Commit Graph

11 Commits

Author SHA1 Message Date
davidmonterocrespo24 cfde1eb27c fix(ci+esp32): unblock backend e2e + bump frontend node heap
Two CI failures landed after PR #196 (esp32-gpio-matrix-cb-callback)
merged. Both are independent and fixed here together.

1) **Backend E2E: ESP32 hangs at bootloader handoff.**
   PR #196 added picsimlab_gpio_matrix_cb which fires on QEMU's
   iothread. The handler did `_emit({...})` for every routing
   change — and the ESP-IDF bootloader writes to gpio_out_sel
   *hundreds* of times during early boot (each peripheral init
   configures its matrix slot). Each emit acquires _stdout_lock
   and writes to the worker→manager pipe. If the manager drains
   even briefly slow, the pipe fills, write blocks, and the
   iothread stalls — symptom: ESP32 reports `entry 0x400805e4`
   then no Arduino setup() output for 75 s.

   Fix: the iothread callback now ONLY mutates the SignalRouter
   snapshot. It never emits. The 10 Hz poll thread
   (_refresh_signal_routing) stays as the sole emitter, so the
   wire-format event stream is unchanged. Benefit of having the
   callback over poll-only is reduced worst-case routing-emit
   latency (next poll tick vs up to 100 ms) and a warmer
   snapshot dict for cheaper poll diffs.

2) **Frontend Tests: Node OOM at end of suite.**
   117 test files run in one forks-pool worker. Several lazy-load
   the ngspice emscripten module (~30 MB), the MixedModeScheduler
   singleton, and other heavy modules whose dispose hooks aren't
   reached because singletons leak across files. Cumulative heap
   pressure exceeds Node's 4 GB default; the worker hits "Ineffective
   mark-compacts near heap limit" AFTER all 1881 tests pass and
   the OOM kill is reported by vitest as "Worker exited unexpectedly
   / Timeout terminating forks worker". This is not a real test
   failure — every individual test passes.

   Quick fix: pass NODE_OPTIONS=--max-old-space-size=8192 to the
   `npm test` step. Long-term, the singletons should add dispose
   hooks that test fixtures call in afterAll(), or the suite
   should shard into multiple `vitest run --shard` invocations.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 16:04:50 +02:00
davidmonterocrespo24 e8557bd25f ci: fix Frontend Tests — use build:docker instead of build
The Vite production build step was running `npm run build` which
includes `tsc -b` in its chain. The repo has ~100 pre-existing strict
TS errors (TS6133 unused vars, TS1294 erasableSyntaxOnly, JSX
intrinsic-element types for custom elements) gated by the separate
`tsc` step above with continue-on-error.

`build:docker` is the script the Dockerfile actually uses to ship
prod — it runs generate:component-svgs + generate:sitemap +
vite build + prerender-seo. It skips `tsc -b` for the same reason
the workflow's `tsc` step is continue-on-error.

Verified locally: 285 SEO pages prerendered, vite build green.
2026-05-15 23:42:52 +02:00
davidmonterocrespo24 07552b5d9e ci: Phase 1d-tests I + K + L — workflow hardening + nightly library-compile
K (frontend-tests.yml reinforced):
  • Matrix node-version: [20, 22] — catches Node-version-specific bugs
  • Cache the 24 MB ngspice WASM by hash — saves ~10s/run
  • `npm run tsc` step (continue-on-error: pre-existing strict errors
    in unrelated test files; tracked but not blocking)
  • `npm run build` — Vite production build smoke catches Rollup/
    Vite-only failures that vitest doesn't see (manualChunks wiring,
    dynamic import paths, asset resolution)
  • `npm run test:coverage` + upload as artifact (Node 22 only)

L (package.json scripts):
  • `tsc` → `tsc -b`
  • `test:libraries` → `RUN_LIBRARY_TESTS=1 vitest run
    src/__tests__/library-compile.integration.test.ts`

I (library-compile nightly):
  • New `.github/workflows/library-compile.yml` — 5 AM UTC cron +
    workflow_dispatch. Not on PRs (slow + external deps).
  • Sets up arduino-cli + caches `~/.arduino15` cores (avr, esp32,
    rp2040 — ~500 MB).
  • New `library-compile.integration.test.ts` — iterates every
    example with `code` + `libraries` + a known FQBN.  For each:
    arduino-cli lib install → write .ino → arduino-cli compile.
    7 examples currently match (epaper-displays).
  • Gated behind RUN_LIBRARY_TESTS=1; default vitest skips the file.

Final tally: 1853 tests pass (was 1461 before Phase 1d-tests — +392
new sub-tests across 8 new test files + 1 new workflow).  Vite build
green (2.68 MB main chunk, unchanged).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-15 23:19:11 +02:00
David Montero Crespo 26c7d50310 fix(ci): two stale-path bugs from the recent refactors
1. Backend test (test_arduino_cli_attinycore.py): the entrypoint script
   was renamed deploy/ → docker/ in commit b736aea but this test still
   pointed at the old path. Update the read_text() call + docstring.

2. Frontend CI (frontend-tests.yml): the cache key
   `frontend-${{ hashFiles('frontend/package-lock.json') }}` was tied to
   a file that has since been gitignored (commit eb9a3ec). hashFiles()
   on a missing file returns the same empty hash forever, so every CI
   run was restoring the same stale node_modules — including the
   symlinks to `file:../third-party/wokwi-elements` that existed before
   the npm migration in commit 531c337. On revalidation, npm tried to
   run wokwi-elements' `prepare` script (`husky install && npm run
   build`), which failed with "husky: not found".

   Drop the cache step entirely; lock files aren't committed so cache
   keys can't be made meaningful without overcomplication. Adds ~30s
   per CI run, but actually correct. Also pass --no-audit --no-fund
   to npm install for cleaner logs.
2026-05-05 11:22:02 -03:00
David Montero Crespo eb9a3ec92f chore: stop committing package-lock.json (cross-platform breakage)
A lock file pins platform-specific native binaries — Rollup, esbuild, swc.
A lock generated on Windows brings @rollup/rollup-win32-x64-msvc but no
Linux variant; a lock generated on Linux does the inverse. The Docker
build kept blowing up with MODULE_NOT_FOUND on rollup/dist/native.js
whenever the lock came from a contributor's non-Linux machine.

Trade-off: we lose npm's transitive-version pinning. Mitigated by:
- package.json caret ranges keep majors stable
- Docker image is rebuilt + retagged per release, so a deployed image
  has a frozen dep set regardless of the lock
- Production uses a pinned upstream commit via velxio-prod's submodule,
  not lock-driven repro
- Dependabot still flags vulnerable transitives via package.json scans

Changes:
- .gitignore: ignore package-lock.json everywhere
- .dockerignore: same (defense-in-depth — never enter build context)
- Dockerfile.standalone: keep `rm -f package-lock.json` as a safety net
  for `docker build` runs from trees with a local lock
- frontend-tests.yml: `npm ci` → `npm install` (npm ci requires a lock)
- Delete the two committed locks (frontend/ + root). The test/* and
  vscode-extension/* locks are left as-is — internal tooling, separate
  install paths, not in the Docker build.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 01:08:23 -03:00
David Montero Crespo 531c337d19 fix(install): unblock self-hosting + drop forced wokwi clones
Resolves several install pain points reported by users (#108, #120) and
removes the obligatory upstream-clone step that confused contributors and
slowed down every Docker build.

Install fixes:
- nginx: server_name → catch-all default_server, drop Debian's stock site
  so reverse-proxied users no longer get the "Welcome to nginx" page.
- entrypoint: auto-generate SECRET_KEY at first boot, persisted under
  data/.secret_key. backend/.env is now optional in docker-compose.yml.
- backend: add greenlet>=3.0.0 (SQLAlchemy async dep that was missing on
  some Python builds — caused uvicorn startup failures on WSL).

Wokwi libs come from npm:
- @wokwi/elements 1.9.2, avr8js 0.21.0, rp2040js 1.3.2 are pinned in
  frontend/package.json. Vite aliases removed.
- Dockerfile.standalone no longer clones avr8js / rp2040js / wokwi-elements
  / wokwi-boards. Frontend stage is just COPY + npm install + build:docker.
- Board SVGs vendored under frontend/public/boards/ (10 deduped against
  existing files, 2 truly new). third-party/wokwi-* clones become reference-
  only credits — generate-component-metadata.ts skips gracefully when absent.

Production config split out:
- docker-compose.prod.yml, deploy/nginx.prod.conf, nginx-host-velxio*.conf,
  update-third-party.bat removed. Production deployment lives in its own
  repo: https://github.com/velxio/velxio-prod (host nginx + HTTPS + backups
  + pinned upstream commit).

Verified locally: 1161 frontend tests pass, build:docker completes clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 00:04:11 -03:00
David Montero Crespo 9cd5061732 refactor: rename wokwi-libs/ → third-party/
The directory grew well beyond Wokwi-only contents: it now hosts
lcgamboa's QEMU fork (qemu-lcgamboa), Espressif's esp32-camera, the
ngspice WASM build, fritzing-parts, picowi, an alternative QEMU
(qemu-esp32), the 100_Days_100_IoT_Projects examples repo, and
Wokwi's own avr8js/rp2040js/wokwi-elements/wokwi-features/wokwi-boards.
"wokwi-libs" was misleading — half the contents have nothing to do
with Wokwi. "third-party/" is the standard convention for vendored
external dependencies.

Mechanical changes:

  Path rename:
    wokwi-libs/ → third-party/
    update-wokwi-libs.bat → update-third-party.bat
    docs/WOKWI_LIBS.md → docs/THIRD_PARTY.md

  Submodule reconfiguration:
    .gitmodules — 4 path= and section names updated
    .git/modules/wokwi-libs/ → .git/modules/third-party/
    each submodule's .git file rewired to ../../.git/modules/third-party/<name>

  Reference updates (~80 files): vite.config.ts aliases, Dockerfile
    COPY paths, GH Actions workflow steps, build_qemu_*.sh, all
    docs/* and test/*/autosearch/* entries that mention the path,
    package-lock.json file: dependencies, .gitignore patterns,
    sitemap.xml + index.html SEO blurbs, scripts/generate-component-*,
    .dockerignore, .idea/vcs.xml. Bulk replaced both `wokwi-libs/`
    (path) and bare `wokwi-libs` (textual mentions in docs/comments).

Verified:
  - npx tsc -b --noEmit produces no new errors related to these paths
  - vite.config.ts aliases now point at ../third-party/avr8js etc.
  - All 4 git submodules (avr8js, rp2040js, wokwi-elements,
    wokwi-features) are linked under third-party/ with their
    worktrees re-populated and config files referencing the new path
  - `grep -r wokwi-libs` returns zero hits outside node_modules,
    .vite, frontend/dist, third-party/ (upstream submodule contents),
    *.pyc caches, and *.dll.pre-camera rollback binaries

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 00:58:57 -03:00
David Montero Crespo 36543e2479 feat: expand SPICE component catalog (fases 9 + 10)
Adds 44 SPICE mappers, 58 custom metadata entries, and 12 visual
Web Components covering logic gates, transistors, op-amps, regulators,
sources, electromechanical parts and integrated-circuit packaging.

Fase 9 — component catalog expansion
------------------------------------
- 7 logic gates (AND/OR/NAND/NOR/XOR/XNOR + NOT) as SPICE B-sources
- 8 multi-input gates (AND/OR/NAND/NOR with 3 and 4 inputs)
- 9 transistors: 5 BJTs (incl. PNP 2N3906/BC557) + 4 MOSFETs (incl.
  P-channel IRF9540/FQP27P06). NMOS refactored from Level=3 W=0.1
  (hangs ngspice) to Level=1 with sane W/L
- 5 op-amps: LM358, LM741, TL072, LM324 with per-chip saturation
  rails + opamp-ideal
- 4 linear regulators (7805, 7812, 7905, LM317) with dropout
- 3 batteries (9V, AA, coin-cell) with realistic ESR
- Signal generator (sine / square / DC)
- 2 Schottky diodes (1N5817, 1N5819) + photodiode (lux-driven
  current source)

Fase 10 — electromechanical + ICs
---------------------------------
- Relay (SPDT): coil + L + S-switch with native hysteresis +
  flyback diode, inverted-control trick for the NC contact
- Optocouplers 4N25 and PC817 (LED + CCCS with CTR=0.5 / 1.0)
- 7 74HC ICs as DIP-14 packages emitting 4 or 6 B-sources per
  component (first mapper pattern emitting multiple device cards)
- 3 flip-flops (D, T, JK) — digital-sim only (edge detection is
  not representable in ngspice .op)
- L293D dual H-bridge motor driver

Infrastructure
--------------
- scripts/component-overrides.json gains a _customComponents[] array
  that lets new Velxio-only parts survive metadata regeneration
  (previously applyOverrides() could only patch wokwi-elements
  components that had already been scanned)
- scripts/generate-component-metadata.ts injects custom entries
  before the patch loop
- New ComponentCategory values: 'logic', 'analog', 'electromech'
- frontend/src/components/DynamicComponent.tsx PASSIVE tracing
  extended from just ['resistor','resistor-us'] to 9 two-terminal
  passives with per-part pin name maps
- New CI workflow test-circuit.yml runs the sandbox on push/PR
- frontend-tests.yml regenerates metadata and fails if committed
  JSON is stale
- Documented 2 new ngspice gotchas in circuit-emulation-gotchas.md:
  unicode in netlist titles silently hangs the parser, and
  MOSFET Level=3 + W=0.1m causes .op to hang
- 164/164 sandbox tests passing in ~9 s (was 88 pre-fase-9)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 22:41:26 -03:00
David Montero Crespo c0b658e6d6 fix: update npm ci to npm install for building libraries and mark submodules as dirty 2026-03-28 22:50:18 -03:00
David Montero Crespo 41d8e25843 feat: enhance admin setup with email validation and update workflows for fresh lib cloning 2026-03-07 00:00:22 -03:00
David Montero Crespo 426c7ab35f feat: establish initial simulator and editor environment with component rendering, wiring, library management, and backend services. 2026-03-04 22:05:23 -03:00