Commit Graph

100 Commits

Author SHA1 Message Date
David Montero Crespo f6f6f43cc2 feat(esp32): velxio_compat.h shim for arduino-esp32 3.x APIs on 2.0.17
A user reported the LEDC PWM RGB example failing to compile. The sketch
calls ledcAttach(pin, freq, resolution) — the one-shot API added in
arduino-esp32 3.x. Our toolchain image pins arduino-esp32 to 2.0.17
(matched to ESP-IDF 4.4.7 + the lcgamboa QEMU ROM), where the API is
the older two-step ledcSetup + ledcAttachPin pair. Sketches written
against 3.x docs hit "ledcAttach was not declared in this scope".

Bumping arduino-esp32 to 3.x means moving to ESP-IDF 5.x, which may
break our QEMU fork. Cheaper fix: ship a compat shim header in the
ESP-IDF project template that defines ledcAttach + ledcAttachChannel
in terms of the 2.x API, gated on `!defined(ledcAttach)` so it
disappears the day we bump.

espidf_compiler.py now injects #include "velxio_compat.h" right after
Arduino.h whether the user explicitly included Arduino.h or we
prepended it ourselves.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 16:32:12 -03:00
David Montero Crespo b373c97377 fix(esp32): suppress ESP-IDF info logs and switch BLE stack to Bluedroid
Two reports rolled into one sdkconfig change:

1. Beta testers reported "weird serial output" across DHT22, Servo+Pot,
   Joystick, WiFi×2, and DualADC examples. The user's Serial.print lines
   came interleaved with internal ESP-IDF chatter:

       I (53306) gpio: GPIO[4]| InputEn: 1| OutputEn: 0| ...
       I (10626) phy_init: phy_version 4791,2c4672b,...

   sdkconfig.defaults didn't pin a default log level so it inherited
   CONFIG_LOG_DEFAULT_LEVEL_INFO. Set WARN (level 2) so only warnings
   and errors leak into the user's serial output. Sketches can still
   esp_log_level_set() per tag at runtime if they want verbose.

2. BLE Advertise example failed to compile. sdkconfig had NimBLE
   enabled, but arduino-esp32 2.0.17's BLEDevice.h targets Bluedroid.
   Switch the stack: enable BT_BLUEDROID + BTDM_CTRL_MODE_BR_EDR_BLE +
   BT_BLE so the standard arduino-esp32 BLE library compiles. Sketches
   that explicitly include NimBLEDevice.h will not compile under this
   config — that's a smaller minority than the BLEDevice.h users.

The runtime side of BLE still depends on stubs in the qemu-lcgamboa
fork; this only fixes the compile path.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 16:29:12 -03:00
David Montero Crespo a3f21a218e fix(esp32): trim flash image before serializing, pad on QEMU attach
Issue #101 reproducer: an ESP32 sketch that pulls in Adafruit_SSD1306 +
Adafruit_GFX produced a "No response from server. Is the backend
running on port 8001?" error in the browser. The compile actually
succeeded backend-side, but the JSON response carrying the firmware
was ~5.5 MB of base64 — the ESP-IDF compiler builds a full 4 MB merged
flash image (mostly 0xFF padding), encodes it whole, and ships it. In
prod that response goes through nginx + Cloudflare, which buffer-fail
or RST the connection on payloads that big — axios then lands in the
"no response" branch with no HTTP status to surface.

Fix: trim the trailing 0xFF padding before serializing, re-pad to a
valid QEMU flash size (2/4/8/16 MB) just before mtd attach. Lossless:
bytes after `last_used` in the merge are 0xFF by construction, so
trim → pad reproduces the original image byte-for-byte.

Numbers from the reproducer (Adafruit_SSD1306 + Adafruit_GFX,
esp32:esp32:esp32 board):
  before: ~5.5 MB JSON response
  after:  539 KB JSON response (10× smaller)

backend/app/services/espidf_compiler.py
  _merge_flash_image now tracks `last_used` across the three placed
  sections (bootloader / partitions / app) and writes only
  flash[:last_used] to merged_flash.bin.

backend/app/services/esp32_flash_image.py (new)
  Shared `pad_to_flash_size(bytes) -> bytes` helper. Rounds up to the
  next valid QEMU flash size with a 4 MB minimum, matches the
  frontend's existing padToFlashSize logic in Esp32MicroPythonLoader.
  Raises ValueError on >16 MB inputs (would indicate a broken upstream
  merge, not anything user-recoverable).

backend/app/services/esp32_lib_bridge.py
backend/app/services/esp32_worker.py
  Both QEMU consumer paths (in-process and subprocess) call
  pad_to_flash_size right after base64.b64decode, before writing the
  tmp .bin that QEMU attaches with `-drive if=mtd,format=raw`.

Verified: smoke test confirms trim → pad → original is byte-exact.
Edge cases covered: small payloads pad up to the 4 MB minimum;
firmwares >16 MB are rejected loudly.

Closes #101

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 14:36:34 -03:00
David Montero Crespo 5f0e7523ed feat(backend): subscription state columns on users table
Adds 4 nullable columns to the users table so deployments that wire an
external billing system (e.g. velxio.dev with Odoo) have a place to cache
subscription status. Self-hosters never write these — defaults are safe
(no paid features unlock).

- models/user.py: is_paid_subscriber (bool, default false), subscription_status
  (str|None), subscription_period_end (datetime|None), odoo_partner_id
  (int|None, indexed).
- main.py: 4 ALTER TABLE statements appended to the legacy_migrations list
  so existing deployments auto-migrate on next boot.
- schemas/auth.py: extend UserResponse with the 3 user-visible fields
  (is_paid_subscriber, subscription_status, subscription_period_end). The
  frontend useAuthStore already persists the whole UserResponse, so these
  surface automatically without any frontend changes upstream.
  odoo_partner_id stays internal — clients don't need it.

Zero behavioural change for existing OSS deployments.
2026-05-05 10:21:37 -03:00
David Montero Crespo edd2ac32d5 feat: add optional extension hooks for private overlays
Three small, backwards-compatible hooks let anyone with private features
(velxio.dev's analytics, custom integrations, paid tiers, …) layer them
on top of the open-source build without forking files.

Backend (app/main.py):
- After standard router registration, try-import an optional `app.pro`
  module exposing `register_pro(app)`. ImportError is silently swallowed
  (the OSS image doesn't ship `app.pro`, so this is a no-op there).

Frontend:
- EditorToolbar: new optional `rightSlot` prop renders extra elements
  after the built-in right-group buttons (mirrors the existing
  `centerSlot` pattern).
- main.tsx: dynamic `import('@pro/index')` gated by VITE_PRO_BUILD env.
  When unset (OSS build), the branch is dead-code-eliminated and no pro
  chunk is emitted.
- vite.config.ts: `@pro` alias resolves to `src/__pro_stub__/` by default.
  Private builds set `VITE_PRO_BUILD=true` and `PRO_OVERLAY_PATH=<path>`
  to point at their real overlay tree.
- src/__pro_stub__/index.ts: 1-line no-op `mountPro` so TypeScript and
  Vite resolvers stay happy in OSS builds.

Verified: `npm run build:docker` succeeds; `npm test` passes 1161/1162;
the OSS bundle (43 MB) contains zero references to `__pro_stub__`,
`@pro`, or `pro/index` (verified via `grep dist/`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 14:03:20 -03:00
David Montero 156d89c61d fix(espidf): include non-utility subdirs for libs without src/ layout
The _should_include filter inside _merge_arduino_libs_to_component
rejected every subdirectory of a library except 'utility/' when the
library lacked a src/ layout. That blocked legitimate header dirs like
Adafruit_GFX_Library/Fonts/, breaking compiles that use any GxEPD2
example with a custom font (#include <Fonts/FreeMonoBold12pt7b.h>).

The earlier filter at the top of the function already excludes
docs/examples/tests/etc. via excluded_dirs, so anything that survives
that check is presumed to be buildable source. Letting all remaining
subdirs through restores Fonts/, gfxfont/, and similar conventional
auxiliary header directories that Adafruit-style libs rely on.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 15:41:53 +02:00
David Montero 38e59cb49d fix(espidf): scan transitive includes recursively for libs with src/ layout
The transitive-include scan in _merge_arduino_libs_to_component used
glob('*.h') on the component dir, which only finds headers at the root.
Libraries with a src/ layout (GxEPD2, ArduinoJson, most modern Arduino
libs) keep their headers under src/<...>, so the scan saw zero headers
and never queued their transitive deps.

Symptom: compiling a sketch that includes GxEPD2_3C.h failed with
'Adafruit_GFX.h: No such file or directory' even though Adafruit_GFX
was installed via the Library Manager — because the BFS never reached
its header from inside GxEPD2_GFX.h.

Switching to rglob('*.h') walks the full directory tree and lets the
BFS pick up Adafruit_GFX, Adafruit_BusIO, and any other transitive
dependency that lives under src/.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 05:52:02 +02:00
David Montero Crespo 531c337d19 fix(install): unblock self-hosting + drop forced wokwi clones
Resolves several install pain points reported by users (#108, #120) and
removes the obligatory upstream-clone step that confused contributors and
slowed down every Docker build.

Install fixes:
- nginx: server_name → catch-all default_server, drop Debian's stock site
  so reverse-proxied users no longer get the "Welcome to nginx" page.
- entrypoint: auto-generate SECRET_KEY at first boot, persisted under
  data/.secret_key. backend/.env is now optional in docker-compose.yml.
- backend: add greenlet>=3.0.0 (SQLAlchemy async dep that was missing on
  some Python builds — caused uvicorn startup failures on WSL).

Wokwi libs come from npm:
- @wokwi/elements 1.9.2, avr8js 0.21.0, rp2040js 1.3.2 are pinned in
  frontend/package.json. Vite aliases removed.
- Dockerfile.standalone no longer clones avr8js / rp2040js / wokwi-elements
  / wokwi-boards. Frontend stage is just COPY + npm install + build:docker.
- Board SVGs vendored under frontend/public/boards/ (10 deduped against
  existing files, 2 truly new). third-party/wokwi-* clones become reference-
  only credits — generate-component-metadata.ts skips gracefully when absent.

Production config split out:
- docker-compose.prod.yml, deploy/nginx.prod.conf, nginx-host-velxio*.conf,
  update-third-party.bat removed. Production deployment lives in its own
  repo: https://github.com/velxio/velxio-prod (host nginx + HTTPS + backups
  + pinned upstream commit).

Verified locally: 1161 frontend tests pass, build:docker completes clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-04 00:04:11 -03:00
David Montero Crespo 9cd5061732 refactor: rename wokwi-libs/ → third-party/
The directory grew well beyond Wokwi-only contents: it now hosts
lcgamboa's QEMU fork (qemu-lcgamboa), Espressif's esp32-camera, the
ngspice WASM build, fritzing-parts, picowi, an alternative QEMU
(qemu-esp32), the 100_Days_100_IoT_Projects examples repo, and
Wokwi's own avr8js/rp2040js/wokwi-elements/wokwi-features/wokwi-boards.
"wokwi-libs" was misleading — half the contents have nothing to do
with Wokwi. "third-party/" is the standard convention for vendored
external dependencies.

Mechanical changes:

  Path rename:
    wokwi-libs/ → third-party/
    update-wokwi-libs.bat → update-third-party.bat
    docs/WOKWI_LIBS.md → docs/THIRD_PARTY.md

  Submodule reconfiguration:
    .gitmodules — 4 path= and section names updated
    .git/modules/wokwi-libs/ → .git/modules/third-party/
    each submodule's .git file rewired to ../../.git/modules/third-party/<name>

  Reference updates (~80 files): vite.config.ts aliases, Dockerfile
    COPY paths, GH Actions workflow steps, build_qemu_*.sh, all
    docs/* and test/*/autosearch/* entries that mention the path,
    package-lock.json file: dependencies, .gitignore patterns,
    sitemap.xml + index.html SEO blurbs, scripts/generate-component-*,
    .dockerignore, .idea/vcs.xml. Bulk replaced both `wokwi-libs/`
    (path) and bare `wokwi-libs` (textual mentions in docs/comments).

Verified:
  - npx tsc -b --noEmit produces no new errors related to these paths
  - vite.config.ts aliases now point at ../third-party/avr8js etc.
  - All 4 git submodules (avr8js, rp2040js, wokwi-elements,
    wokwi-features) are linked under third-party/ with their
    worktrees re-populated and config files referencing the new path
  - `grep -r wokwi-libs` returns zero hits outside node_modules,
    .vite, frontend/dist, third-party/ (upstream submodule contents),
    *.pyc caches, and *.dll.pre-camera rollback binaries

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 00:58:57 -03:00
David Montero Crespo 977308ef80 fix(spi-batch): add 50ms safety flush so frames keep arriving
User reported only 2 frames rendering after the SPI batching change.

Diagnosis: the previous flush triggers (CS-line HIGH, buffer >=4096)
were both event-driven. Adafruit_ILI9341's ESP32 backend manages CS
via digitalWrite — i.e. through the GPIO peripheral, NOT the SPI
peripheral's hardware CS pin. So the CS-line-HIGH event from
picsimlab_spi_event NEVER fires for this driver. The buffer only
flushes when it hits 4096 bytes.

Frame 1: 38 400 bytes from drawRGBBitmap → 9 flushes at 4096-byte
boundaries → last 192 bytes stay in the buffer. Status bar adds
some bytes too → maybe one more flush.

Frame 2: same. But by frame 3 the firmware is running ahead of the
flush rhythm and somehow the buffer pattern wedges in a state where
no flush completes (likely a partial buffer that sits between
transactions while the firmware briefly waits on the next fb_get).
Hard to reproduce deterministically — but the symptom matches.

Fix: add a 50 ms periodic flush thread. Independent of any event,
it acquires the lock and flushes whatever's pending. Bounds the
worst-case latency at 50 ms (= 20 fps ceiling, more than enough for
the emulator).

Triple-trigger now:
  1. CS HIGH (fast path for hardware-CS drivers)
  2. Buffer >= 4096 (safety for big transactions)
  3. 50 ms timer (catches GPIO-CS drivers, prevents stalls)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 00:31:16 -03:00
David Montero Crespo d4d015c25d perf(spi): batch SPI bytes per WS message — ~50× faster TFT in emulator
User reported the ESP32-CAM + ILI9341 live preview at ~1 frame/min.
Profile: 80×60 preview pushes 9600 SPI bytes per drawRGBBitmap, and
each byte was emitting a full {type:'spi_event'} JSON message over
the worker→backend→WS→frontend pipeline. Per-byte overhead ~150-200µs
in Python (json.dumps + sys.stdout.write+flush dominates) plus
asyncio + WS dispatch. Net: 1.5-2 sec/frame minimum, much worse with
GIL contention.

Fix: buffer MOSI bytes in the worker and emit a single base64-encoded
`spi_batch` message when CS goes HIGH (transaction ended) or the
buffer crosses 4 KiB. ~9600 events/frame collapse to ~3 messages.

  backend/app/services/esp32_worker.py:_on_spi_event
    - Add _spi_byte_buf bytearray + threading.Lock
    - On op==0x00 (byte): append; flush early if buf >= 4096
    - On op==0x01 (CS change): flush buffer, then emit the CS event
      via the legacy spi_event channel (ePaper / custom chips that
      observe CS still get it).

  frontend/src/simulation/Esp32Bridge.ts
    - New 'spi_batch' message handler decodes b64 and replays each
      byte through the existing onSpiByte callback. Parts that
      subscribed via simulator.spi.onByte don't notice the protocol
      change. The 'spi_event' branch still handles CS changes plus
      legacy single-byte payloads for backwards compat.

Now that 38 KB/frame is cheap, restore preview to 160×120 + JPEG
quality 0.35 in the gallery example. Real measured speedup: ~50× on
the QVGA preview demo. Real hardware was never affected — it runs
SPI at 80 MHz and pushes the bitmap in ~4 ms either way.

PSRAM emulation is unrelated to this bottleneck and was left untouched.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-03 00:25:06 -03:00
David Montero Crespo c068612077
Merge pull request #137 from davidmonterocrespo24/esp32-cam
Esp32 cam
2026-05-02 22:40:06 -03:00
David Montero Crespo 442be32a6f feat(esp32-cam): real webcam emulation verified end-to-end
User confirmed the FULL pipeline works with their laptop webcam at
QVGA quality 0.6 — frontend → WS → backend → worker → DLL → I²S →
firmware → esp_camera_fb_get() → user sketch.

  [Frame #1]  8192 bytes  320x240  fmt=4
    ├─ SOI (FF D8 FF):     ✓ at offset 0
    ├─ EOI (FF D9):        ✓ at offset 8190
    └─ First 16 bytes:     FF D8 FF E0 00 10 4A 46 49 46 …
                                                (JFIF, real webcam)
  Stats: 10/10 Valid JPEGs at ~3.5 fps.

Changes:
- wokwi-libs/qemu-lcgamboa @ e4321d1 (picsimlab-esp32):
    EOFS_PER_FRAME 6→8, inject_eoi_now flag for EOI injection on the
    last EOF of each VSYNC burst. Handles JPEGs of arbitrary size by
    forcing FF D9 at offset 8190 — JPEG decoders tolerate the
    truncation gracefully.
- backend/esp32_worker.py: throttled trace log every 30 frames
    (`camera_frame #N received (NNNN bytes)`) so users can confirm
    the frontend → worker leg is alive without flooding the log.
- test/test-esp32-cam/autosearch/14: documented bug #9 (the 9th and
    final silent bug — real webcam JPEGs exceed the deliverable byte
    budget) with full forensic trace + final architecture diagram.

The emulation now handles ANY user webcam → ESP32-CAM use case end
to end. Standard upstream esp_camera_init() / esp_camera_fb_get()
sketches work without modification.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 22:05:37 -03:00
David Montero Crespo e73c1d341c feat: ESP32-CAM emulation with webcam frame bridge
First open-source end-to-end emulation of the AI-Thinker ESP32-CAM
in QEMU, paired with a browser webcam → firmware bridge so users can
develop camera sketches without hardware. Status: esp_camera_init()
returns ESP_OK; OV2640 chip-id verifies (PID/VER/MIDH/MIDL exactly
match the datasheet); GPIO 25 VSYNC NEGEDGE interrupt enabled by
the upstream driver. Final piece (cam_task accepting frames) is in
progress — descriptor walker fix landed in this commit.

Backend (Python/FastAPI):
- simulation.py: camera_attach/frame/detach WS handlers
- esp32_worker.py: ctypes binding to velxio_push_camera_frame +
  feature-detection fallback for older DLLs
- esp32_lib_manager.py: forward camera commands to the worker stdin
- esp-idf-template/main/CMakeLists.txt: esp32-camera headers added
  via add_prebuilt_library + REQUIRES driver (resolves i2c_master_*
  symbols). LED_BUILTIN=2 fallback for sketches that hardcode it.

Frontend (React/TS):
- EditorToolbar.tsx: ESP32-CAM (and the rest of the ESP32 family)
  added to isQemuBoard list — Run button now starts the QEMU bridge
  for these boards instead of falling through to the AVR path
- useWebcamFrames.ts: getUserMedia → OffscreenCanvas →
  toBlob('image/jpeg') → base64 → WS at ~10 fps
- CameraToggle.tsx: header button with status colors + frame counter
- SimulatorCanvas.tsx: render CameraToggle for esp32-cam boards
- Esp32Bridge.ts: sendCameraAttach/Frame/Detach + chunked btoa
- useSimulatorStore.ts: diagnostic log on compileBoardProgram
- components-metadata.json: regen including esp32-cam component

Submodule pointer:
- wokwi-libs/qemu-lcgamboa → ff8eee0 (camera devices commit on
  davidmonterocrespo24/qemu-lcgamboa branch picsimlab-esp32)

Investigation + tests in test/test-esp32-cam/:
- 13 autosearch markdown docs (overview, SOTA, OV2640 spec, DVP/I2S
  spec, build blueprint, blockers resolved, descriptor walker fix)
- 5 sketches (camera_init, sccb_probe, dma_smoke, frame_roundtrip,
  webcam_demo) + 8 live + WS regression tests
- README with the user-facing flow

.gitignore:
- libqemu-*.dll.{pre-camera,new,bak} (rollback points, regenerated)
- wokwi-libs/esp32-camera/ (clone consumed by arduino-esp32 path,
  not part of this repo)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-02 19:29:01 -03:00
David Montero Crespo 29a31d8e0a
Merge pull request #134 from ZhadowValker/fix/espidf-library-structure
fix: Preserve library directory structure in ESP-IDF component conversion
2026-05-02 19:16:46 -03:00
ZhadowValker cc43a956ba feat: Add library version management and uninstall functionality
Backend:
- Add version field to InstallLibraryRequest
- Add fallback and requested_version to InstallResponse
- Add DELETE /api/libraries/uninstall endpoint
- Enhance install_library() for versioned installs (LibName@version)
- Add semver validation and fallback logic
- Add uninstall_library() method
- Fix _parse_version() to reject non-numeric version parts

Frontend:
- Update installLibrary() with optional version parameter
- Add uninstallLibrary() and resolveLibraryVersion() helpers
- Add version selector dropdown in Library Manager
- Add UNINSTALL button for installed libraries
- Show fallback messages when requested version unavailable
- Add parseLibSpec() and version badges in InstallLibrariesModal
2026-05-02 12:54:09 +05:30
ZhadowValker 79cfd8197d fix: Apply library structure preservation to _merge_arduino_libs_to_component
_create_idf_component was fixed but NOT _merge_arduino_libs_to_component.
The latter was still flattening library files, causing ArduinoJson compilation to fail
with 'src/ArduinoJson.h: No such file or directory'.

Changes:
- Replace flat copy with directory-structure-preserving copy
- Add exclusion logic for non-buildable directories (examples, tests, docs)
- Generate INCLUDE_DIRS from actual directory structure
- Track files by relative path to prevent name collisions

This ensures libraries with src/ layouts (ArduinoJson, etc.) compile correctly.
2026-05-02 11:03:04 +05:30
ZhadowValker 4ff1502b96 refactor: preserve library directory structure in ESP-IDF component conversion
- Maintain original library layout (src/, utility/) instead of flattening files
- Add exclusion logic for non-buildable directories (examples, tests, docs, CI)
- Dynamically generate INCLUDE_DIRS from actual directory structure
- Add validation to ensure buildable source files exist before proceeding
- Support both flat and src-based library layouts
- Fix path separator normalization for cross-platform compatibility

This improves compatibility with complex Arduino libraries that rely on
specific directory structures and relative includes.
2026-05-02 10:54:24 +05:30
David Montero Crespo 6a88375bc0 feat: persist multi-board projects + add auto-save
The project save/load pipeline only persisted a single `board_type`, so
multi-board workspaces silently lost every board except the active one
on save, and wires referencing the dropped boards' IDs orphaned to the
canvas corner on reload. An audit of the production backup found 74/306
projects (24%) with at least one orphaned wire and 174/301 non-trivial
projects whose code was still the default Blink template — strong signal
that users save once and never re-save.

Backend
- Add `boards_json` column on `projects` with idempotent ALTER TABLE in
  the lifespan migration list.
- New `FileGroup` schema + `file_groups` array on
  ProjectCreate/Update/Response. Legacy `files`/`code` kept for back-compat.
- `project_files.py` now uses `{pid}/{groupId}/{filename}` subdirs via
  `read_groups`/`write_groups`. Legacy flat layouts are auto-promoted on
  read; legacy single-list `files` only updates the active group, leaving
  other boards' files intact.
- `_persist_files_from_body` honors file_groups → files → code priority.

Frontend
- `useSimulatorStore.addBoard` accepts an optional `explicitId` so
  saved board IDs can be restored verbatim (wires reference IDs literally).
- New `loadProjectState({boards, fileGroups, components, wires,
  activeBoardId})` action: tears down current boards, recreates from the
  payload, restores file groups atomically, recalculates wire positions
  on the next frame, and refreshes the Interconnect.
- `useEditorStore.replaceFileGroups` for atomic multi-group restore.
- `SaveProjectModal` and `ProjectByIdPage`/`ProjectPage` now go through
  `buildSavePayload` / `buildLoadPayload` (handles pre-backfill projects
  by synthesising a default board from `board_type`).

Auto-save (#useAutoSaveProject hook)
- 2.5s debounced silent PUT triggered ONLY when an authenticated user
  has a `currentProject` with a UUID. State hash detects real changes
  vs. UI-only churn; baseline is reset on project load so the just-loaded
  state isn't immediately re-saved.
- `beforeunload` flush via `fetch keepalive: true` (supports PUT +
  credentials, survives unload).
- Compact status indicator in `AppHeader` (idle/dirty/saving/saved/error).

Backfill script (one-off, idempotent)
- `backend/scripts/backfill_boards_2026_05.py` populates `boards_json`
  for legacy projects. Heuristic per project, based on which board IDs
  the wires reference:
    Case A — wires only ref 'arduino-uno' but board_type ≠ uno:
             rename id→board_type and rewrite wire endpoints.
    Case B — single-board normal: keep verbatim.
    Case C — multi-board: recreate one board per distinct ref, infer
             kind by stripping trailing -N suffix.
  Also moves any flat files into the active board's group subdir.
  Stdlib-only, runs from host or `docker exec`.

Docker
- `Dockerfile.standalone` now copies `backend/scripts/` into the image
  so the backfill is callable via `docker exec velxio-app python
  /app/scripts/backfill_boards_2026_05.py --apply`.

Verified locally on the restored production backup (363 projects):
33 Case A, 316 Case B, 14 Case C, 135 wire endpoints renamed, 0 orphans.
Re-running the script after apply skips all 363 (idempotent).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-01 13:43:33 -03:00
David Montero Crespo d346e89b92 Refactor ESP32 library management and add regression tests for issue #129
- Update esp32_lib_manager.py to dynamically set library extensions based on the platform (Linux, Windows, macOS).
- Update submodule references for qemu-lcgamboa and wokwi-elements.
- Add documentation on ESP32 Arduino runtime crashes related to cache disable during WiFi/BT initialization.
- Introduce regression tests for the ESP32-CAM blink issue, ensuring the user sketch matches the reported problem.
- Implement an IRAM-safe blink sketch to confirm the regression is due to the Arduino runtime.
- Create a comprehensive test suite to cover various layers of the simulation and compilation process.
2026-04-30 23:49:46 -03:00
David Montero Crespo 2e4c470f75 feat: add support for UC8159c (ACeP 7-colour) display
- Implement UC8159cDecoder for handling 7-colour ACeP panels.
- Introduce painting functions for UC8159c frames in EPaperPart.
- Update EPaperPart to handle both SSD168x and UC8159c frame types.
- Add integration tests for EPaperPart and UC8159cDecoder.
- Create example sketch for 5.65" ACeP 7-colour panel.
- Enhance error handling in test cases for library dependencies.
2026-04-30 00:27:40 -03:00
David Montero Crespo bcc7129aa3 feat: Add support for SSD168x ePaper panels
- Introduced EPaperPanels.ts to define configurations for various ePaper panels including dimensions, refresh rates, and controller details.
- Implemented SSD168xDecoder.ts to handle the decoding of SPI commands for the SSD168x family of ePaper displays.
- Created EPaperPart.ts to manage the simulation of ePaper panels, integrating with the existing simulator architecture and handling events.
- Added example sketches for 2.13", 2.9", 4.2", and 7.5" ePaper displays, demonstrating basic functionality and text rendering.
- Ensured compatibility with AVR, RP2040, and ESP32 platforms, with appropriate pin configurations for each.
2026-04-29 22:23:00 -03:00
David Montero Crespo 89c9d7c6c6 Implement TCP and UDP NAT services for chip-initiated connections
- Added tcp_nat.py to handle TCP NAT, implementing the three-way handshake, data flow, and connection state management.
- Introduced udp_nat.py for UDP NAT, managing chip-initiated datagrams and maintaining flow state.
- Created integration tests in test_picow_net_bridge.py to validate the functionality of the TCP and UDP NAT implementations, including ARP, DHCP, ICMP, and DNS interactions.
2026-04-29 08:43:41 -03:00
David Montero Crespo 175b248108 feat(epaper): Add SVG layouts and emulation plan for ePaper panels
- Introduced SVG layout dimensions for Phase 1 (B/W mono) and Phase 2 (colour) ePaper panels, detailing active areas, bezels, and pin layouts.
- Developed a phased emulation plan outlining the architecture and deliverables for different panel types, including SSD168x and UC81xx.
- Created a canonical "Hello, World!" sketch for the 1.54" ePaper panel, ensuring compatibility across ESP32, Raspberry Pi Pico, and Arduino Uno.
- Implemented a pure Python SSD168x decoder to validate SPI command sets and framebuffers against specifications.
- Added tests for compiling the hello-world sketch across supported boards and for the SSD168x protocol to ensure correct framebuffer behavior.
2026-04-29 02:33:59 -03:00
David Montero Crespo 641ac8c1de Add comprehensive tests for Cyw43Emulator functionality and lifecycle
- Implemented handshake tests to validate initial bus state and register responses.
- Created end-to-end tests for Pico W LED blinking using MicroPython firmware.
- Added SDPCM framing tests to ensure proper encoding and decoding of control frames.
- Developed IOCTL tests to verify command responses and state changes in the emulator.
- Established a full lifecycle test for WiFi operations, including scanning, connecting, and packet handling.
- Introduced TypeScript configuration for test files to ensure compatibility and strict type checking.
2026-04-29 00:21:26 -03:00
David Montero Crespo 7f2014bef7 Add ESP32 chip demos and comprehensive tests for I2C, SPI, and UART interactions
- Implemented `esp32_spi_chip_demo.ino` to demonstrate SPI communication with a 74HC595 shift register.
- Created `esp32_uart_chip_demo.ino` for UART loopback testing with ROT13 transformation.
- Added Python tests for compiling chips and sketches, ensuring valid WASM output and successful compilation for various board families.
- Developed end-to-end tests for ESP32 with custom chips using I2C and SPI, validating synchronous communication through the backend.
- Introduced GPIO bridge tests to verify serial communication and GPIO state changes.
- Ensured all tests validate the expected behavior of the custom chips and their interaction with the ESP32 firmware.
2026-04-28 19:24:39 -03:00
David Montero Crespo 63896e2049 feat(activity): add user daily activity metrics and modal for detailed project interaction 2026-04-26 19:39:45 -03:00
David Montero Crespo ba63aba4dc feat(env): add example environment configuration file for backend 2026-04-26 18:03:04 -03:00
David Montero Crespo 5bf3a3d5ed feat(metrics): add usage analytics dashboard with country tracking
Track per-user, per-project, and per-board usage to inform pricing tier
decisions. Adds an admin dashboard with KPIs (DAU/WAU/MAU, totals,
success rate), time-series charts for compiles/runs, board family +
FQBN breakdowns, "board diversity" pie chart (key freemium signal),
top users/projects, and per-country breakdown via Cloudflare's
CF-IPCountry header. Admin can now also view private projects.

Backend:
- New UsageEvent table (append-only event log with user_id, project_id,
  event_type, board_fqbn/family, country, error_kind, duration_ms)
- Aggregate counters on User (total_compiles/runs/errors, last_active,
  signup_country, last_country) and Project (compile/run/update counts,
  last_compiled/run timestamps) kept in sync by MetricsService for O(1)
  dashboard reads
- 10 admin endpoints under /api/admin/metrics/{overview, timeseries,
  boards, board-diversity, top-users, top-projects, countries,
  users/{id}, projects/{id}}
- POST /api/metrics/run for client-side run telemetry
- Country detection via cf-ipcountry header (no DB / no API calls)
- Auto-migrations in lifespan for legacy DBs

Frontend:
- recharts-powered Dashboard tab with KPI cards and 4 charts
- New Boards tab with per-family + per-FQBN breakdown
- Country column with flag emoji on Users tab
- Top countries card on Dashboard
- compileCode now forwards project_id; Run button reports via WS

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-25 19:47:40 -03:00
David Montero Crespo 9cc9cfebd6 Add end-to-end tests for ammeter, voltmeter, and capacitor charging behavior
- Implement `ammeter-waveform.test.ts` to validate AC readings from a sine wave source.
- Create `capacitor-charge-transient.test.ts` to test the charging response of an RC circuit driven by a microcontroller pin.
- Introduce `esp32-rectifier-integration.test.ts` for testing rectifier behavior using QEMU and ESP32.
- Add helper functions in `esp32RectifierE2E.ts` for the rectifier test harness.
- Develop `voltmeter-waveform.test.ts` to ensure correct AC and DC readings from a sine wave source.
- Implement unit tests for waveform statistics in `waveform-stats.test.ts` to validate RMS, mean, peak, and interpolation functions.
- Create `waveformStats.ts` to provide statistical functions for time-domain waveform analysis.
2026-04-21 02:17:30 -03:00
David Montero Crespo f5ef107eaa feat: implement asyncio exception handler and update entrypoint script for process management 2026-04-16 00:57:30 -03:00
David Montero Crespo 4d6dc25ec7 feat: add BMP280 sensor component and circuit preview
- Implemented Bmp280Element as a custom web component for the BMP280 barometric sensor, including SVG representation and pin configuration.
- Created CircuitPreview component to render circuit thumbnails using SVGs of components, including support for various boards and components.
- Added a script to generate SVG files from wokwi-elements, ensuring proper formatting and structure for reliable rendering.
- Introduced a test HTML generation script to visualize component SVGs.
2026-04-14 17:27:30 -03:00
David Montero Crespo 74e25eb4b2 feat: Update MicroPython firmware handling for ESP32; add end-to-end test and mark subproject commits as dirty 2026-04-12 23:55:11 -03:00
David Montero Crespo c736e6e789 feat: Add compiler flag for BitOrder restoration in Arduino CLI; mark subproject commits as dirty 2026-04-12 00:23:06 -03:00
David Montero Crespo 830aa138ae feat: Update dependencies in requirements.txt; mark subproject commits as dirty and add pip installation log 2026-04-11 15:45:50 -03:00
David Montero Crespo 2ba8020438 feat: Enhance ESPIDFCompiler library resolution logic; add support for dynamic library detection and patching in CMakeLists.txt
refactor: Update wiring examples for E32 OLED integration; correct pin mappings for VCC, GND, DATA, and CLK
test: Improve unit tests for ESPIDFCompiler; add scenarios for library resolution and CMake patching
chore: Mark subproject commits as dirty for wokwi-libs
2026-04-11 15:25:40 -03:00
David Montero Crespo 7971743174 Add unit tests for I2C slave devices, MCP tools, and WiFi/BLE status parser
- Implement tests for BMP280, DS1307, DS3231, I2CWriteSink, and MPU6050 slaves in test_i2c_slaves.py.
- Create test suite for Velxio MCP server tools in test_mcp_tools.py, covering Wokwi utilities and circuit management functions.
- Add tests for parsing WiFi and BLE serial output in test_wifi_status_parser.py, ensuring correct status events are captured.
2026-04-10 23:39:51 -03:00
David Montero Crespo d2e1e04def Add comprehensive documentation for ESP32 GPIO sensor simulation
This commit introduces a detailed markdown document outlining the process of simulating DHT22 and HC-SR04 sensors on the ESP32 platform using Velxio's QEMU fork. The documentation covers the context of the simulation, key callbacks, problems encountered, and solutions implemented for both sensors. It includes architectural details, end-to-end testing procedures, and guidelines for adding new GPIO-timed sensors. The aim is to provide maintainers with a thorough understanding of the GPIO logic and the challenges faced during development.
2026-04-10 22:51:56 -03:00
David Montero Crespo 46e459f51b Refactor I2C slave tests for ESP32: update event handling and improve accuracy of ACK/NACK responses; add full end-to-end test for MPU-6050 I2C simulation; update components metadata timestamp; mark subproject commits as dirty for wokwi-libs. 2026-04-09 15:06:39 -03:00
David Montero Crespo 5795b1d506 Fix MPU6050Slave I2C handling and add comprehensive tests
- Updated the threshold for switching to data mode in MPU6050Slave from 2 to 3 WHO_AM_I reads to ensure correct chip identification.
- Enhanced comments in the code to clarify the sequence of I2C events during initialization.
- Added a new test file `test_mpu6050_emulation.py` to validate the MPU6050Slave state machine and ensure it handles the full Adafruit_MPU6050::begin() event sequence correctly.
- Updated existing tests to reflect the changes in the I2C handling logic.
- Modified `components-metadata.json` to update the generated timestamp.
- Marked submodules `rp2040js` and `wokwi-elements` as dirty to reflect local changes.
2026-04-09 02:04:41 -03:00
David Montero Crespo 4667c8bb3b feat: Enhance MPU6050Slave I2C handling with improved WHO_AM_I read tracking; update tests for clarity 2026-04-08 08:48:14 -03:00
David Montero Crespo ca6520e48d feat: Update I2C event handling and improve MPU-6050 slave emulation; enhance README and tests 2026-04-08 00:02:54 -03:00
David Montero Crespo 0f2c39f23b feat: Add I2C sensor support and implement ESP32 I2C slave emulation
- Introduced I2C_SENSOR_MAP for pre-registering I2C sensors in the simulator store.
- Implemented I2C slave state machines for MPU6050, BMP280, DS1307, and DS3231 sensors in esp32_i2c_slaves.py.
- Added unit tests for I2C slave functionality covering BMP280, DS1307, DS3231, and I2CWriteSink.
- Updated the simulator store to handle I2C address resolution and sensor data management.
- Marked submodules as dirty in wokwi-libs for rp2040js and wokwi-elements.
2026-04-08 00:02:43 -03:00
David Montero Crespo 689f8e71db feat: Add I2C slave emulation for MPU-6050 and BMP280 sensors
- Implemented _MPU6050Slave and _BMP280Slave classes for I2C communication.
- Enhanced main function to register these sensors and handle I2C events.
- Updated sensor management to support MPU-6050, BMP280, DS1307, DS3231, SSD1306, and PCF8574.
- Added frontend examples for BMP280 weather station and SSD1306 OLED display.
- Modified Esp32Bridge to handle new I2C transaction events.
- Updated ProtocolParts to support ESP32 path for I2C devices.
- Enhanced useSimulatorStore to manage I2C transaction listeners.
2026-04-07 15:43:09 -03:00
David Montero Crespo 9761aad0be feat: enhance Arduino library handling by detecting external libraries and creating IDF components, add tests for library resolution logic 2026-04-07 14:59:51 -03:00
David Montero ef299ba7fa fix: align ESP32 WiFi SSID/channel with QEMU access_points[] array
Two fixes for ESP32 WiFi not connecting in production:

1. espidf_compiler.py: Change WiFi normalization from 'Velxio-GUEST' on
   channel 6 to 'Espressif' on channel 5. The lcgamboa QEMU binary
   downloaded from GitHub Releases only contains the original three APs:
   PICSimLabWifi (ch1), Espressif (ch5), MasseyWifi (ch10). Channel 6
   had no matching AP, so the beacon timer's channel-match condition never
   fired → firmware scanned forever and never connected.

2. esp32_worker.py: Redirect fd 1 to /dev/null before loading QEMU so
   raw UART bytes from QEMU's -nographic mux don't corrupt the JSON
   event pipe. The real pipe fd is saved and sys.stdout is rebound so
   _emit() continues to work. This also prevents stdout pipe back-pressure
   from stalling qemu_main_loop() (and thus REALTIME timers).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 06:11:29 +02:00
David Montero 32acf80e0d fix: propagate has_wifi from compiler to startBoard for reliable WiFi detection
Frontend WiFi detection via file-content scanning was unreliable because
fileGroups[board.activeFileGroupId] could be an empty array (not null),
bypassing the ?? fallback to editorState.files.

Fix: the ESP-IDF compiler now returns has_wifi:bool in its compile response.
The frontend stores this on the BoardInstance and uses it in startBoard()
instead of scanning file contents. The file-content scan is kept as a
fallback for boards that haven't been compiled in this session.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 22:14:40 +02:00
David Montero 1298be72b2 fix: add RISC-V toolchain paths to build env for ESP32-C3 compilation
_build_env() on Linux only set IDF_TOOLS_PATH but never added the tool
binary directories to PATH, so cmake could not find riscv32-esp-elf-g++
when compiling for ESP32-C3. Also improve ninja failure logging to show
stdout (where build errors actually appear) instead of empty stderr.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-05 20:55:38 +02:00
David Montero Crespo b0fe5b5d14 feat: enhance logging for library loading and WiFi progress; update subproject commits 2026-04-05 00:49:25 -03:00
David Montero Crespo 7ccd5a7b63 feat: enhance ESP32 boot stability with deterministic instruction counting and mark subprojects as dirty 2026-04-05 00:16:40 -03:00