Move the CYW43439 (Pico W) WiFi emulation out of the open-source tree so it
can ship as a paid feature in a private overlay. OSS keeps a plain Pico W
(no WiFi); the overlay registers the cyw43 protocol + backend network stack
at runtime via generic seams.
Frontend:
- Add simulation/PioPeripheral.ts: a generic "PIO bus peripheral" seam
(feedWord / inDiscardableWriteData / resetFraming / hostWakeLevel /
onHostWake / onSimulationStart). No factory is installed in OSS, so
createPioPeripheral() returns null and a Pico W simulates as a plain Pico.
- RP2040Simulator: keep the fragile PIO-FIFO plumbing (it must re-run after
loadMicroPython swaps the chip) but drive it through PioPeripheral instead
of an inlined cyw43 import (attachCyw43 -> attachPioPeripheral, etc.).
- useSimulatorStore: generic attach/detach + setBoardWifiStatus; drop the
cyw43 bridge map.
- MicroPythonLoader: add registerFirmwareVariant() so an overlay can add the
RPI_PICO_W build; remove the OSS pico-w config + bundled .uf2.
- Delete simulation/cyw43/ (moved to the overlay).
Backend:
- core/hooks.py: add generic register_ws_sim_handler / dispatch_ws_sim_message
and register_gateway_proxy / dispatch_gateway_proxy seams.
- simulation.py: route start_picow / stop_picow / picow_packet_out through the
ws_sim_handler hook (the overlay handles + gates them).
- iot_gateway.py: resolve the Pico W gateway through the gateway_proxy hook.
- Delete services/picow_net/ + picow_net_bridge.py (moved to the overlay).
Tests: move the cyw43/picow suites to the overlay; update RP2040Simulator
mock stubs to attachPioPeripheral.
Two real-world failures hit the Pico W gateway that the headless e2e
(node fetch, tiny request, fast timing) didn't surface:
- A browser sends KILOBYTES of headers (cookies, User-Agent, sec-*).
Forwarded verbatim, the request overran the chip's small recv()
(e.g. recv(1024)); lwIP then RST the connection on close-with-unread-
data, crashing blocking-socket sketches with ECONNRESET. Now we forward
a MINIMAL request (method, path, Host, Connection: close, and
Content-Type/Length for bodies) — nothing a tiny server can choke on.
- After a gateway request completed we dropped the connection immediately,
so a late chip segment (retransmitted FIN / trailing ACK) no longer
matched and fell through to the chip-initiated NAT, which RST it. Add a
short TIME_WAIT: keep the connection briefly and re-ACK late segments so
the chip closes cleanly, never RSTing it.
ESP32 web-server examples are reachable from the browser via
/api/gateway/<client_id>/ (QEMU slirp hostfwd). The Pico W server lives
in the browser-side lwIP, so there was no inbound path: visiting the
chip's IP did nothing.
Add the mirror of tcp_nat.py: tcp_inbound.TcpInbound originates a TCP
connection INTO the chip over the WebSocket bridge (SYN -> SYN+ACK ->
ACK -> request -> response -> FIN), so the backend can fetch a page the
sketch serves on 10.13.37.42:80 and hand it back to the browser.
- bridge.py routes chip TCP segments addressed to a gateway-opened
connection to TcpInbound (before the chip-initiated NAT, which would
RST them); exposes http_into_chip() + ensure_chip_mac() (primes the
chip's gateway ARP).
- iot_gateway.py: same /api/gateway/<client_id>/ route now falls through
to the Pico W bridge when there's no ESP32 instance, builds a raw
HTTP/1.1 request, and parses the chip's response. Same plan gate, same
URL shape — the browser sees no difference between ESP32 and Pico W.
Validated end to end (real RP2040 emulator serving an HTTP page ->
gateway returns it) plus 6 unit tests for the TCP state machine,
response parsing and ARP priming.
Add a working microSD card part backed by a FAT16 image, following the
Wokwi storage model: the project's own workspace files are auto-copied
onto the card (free), and an optional "SD Card" panel uploads extra
files (gated as a paid feature by the velxio.dev overlay; OSS default
allows it).
Frontend (in-browser AVR / RP2040):
- ProtocolParts.ts: rewrite the microsd-card part from a handshake stub
into a real SD-over-SPI device (reply-first Ncr timing, SDSC byte
addressing, single/multi-block read+write, CSD/CID, full CMD set).
- utils/fatImage.ts: dependency-free FAT16 super-floppy builder (8.3 + LFN).
- utils/sdCardFiles.ts: assemble the card image from workspace files plus
uploaded files; base64 helpers.
- components/simulator/SdCardPanel.tsx + ComponentPropertyDialog: upload UI.
- DynamicComponent + useSimulatorStore: build and inject the image on run.
- lib/proSdCardGate.ts: overlay-installable gate for the upload action.
- data/examples-storage-microsd.ts: Arduino Uno + ESP32 gallery examples.
Backend (ESP32 via QEMU):
- services/esp32_sd_slave.py: synchronous SD-over-SPI slave (Python port of
the browser part) with a sparse backing store, idle-state R1 tracking and
real CRC16 on data blocks when the host enables CRC (CMD59) -- both
required by ESP-IDF's sdspi driver.
- esp32_worker.py: route SPI bytes to the slave (returns MISO synchronously)
and feed write-only bulk transfers.
- esp32_lib_manager.py + routes/simulation.py: forward the FAT image
(sd_card.image_b64) from the start config into the worker.
Tested:
- frontend: protocol-parts, fat-image, sd-card-gate and microsd-real-firmware
(real Arduino SD.h on avr8js) -- 86 passing.
- backend: test_esp32_sd_slave (10) covering the ESP-IDF init sequence and
CRC16; validated end to end by running a real SD.h sketch in libqemu-xtensa
(mount, directory listing, read and write-readback).
arduino-esp32's WiFiClientSecure/ssl_client.cpp wraps its ENTIRE body
(start_ssl_client, ssl_init, send_ssl_data, ...) in
#if !defined(MBEDTLS_KEY_EXCHANGE_SOME_PSK_ENABLED) ... #else <body> #endif
ESP-IDF's mbedtls defaults the PSK key-exchange modes OFF, so the object
compiled empty and any sketch using WiFiClientSecure — including
HTTPClient.begin(url), which links the secure client even for http:// —
failed to link with "undefined reference to start_ssl_client". Commenting
out begin() let the optimizer drop the unused client, which is why it
"compiled when commented".
- sdkconfig.defaults.in: enable the PSK key-exchange ciphersuites
(CONFIG_MBEDTLS_PSK_MODES + the four KEY_EXCHANGE_*PSK), matching
arduino-esp32's own sdkconfig.
- espidf_compiler: ESP-IDF only seeds sdkconfig from sdkconfig.defaults when
sdkconfig is ABSENT. Persistent build dirs live in the build volume and
keep a stale sdkconfig across image rebuilds, so the new CONFIG_* would
never reach kconfig. Drop the generated sdkconfig when the rendered
defaults change so it re-seeds on configure.
- test: assert the rendered sdkconfig enables PSK.
Verified end-to-end: the reported WiFi+HTTPClient sketch now compiles to a
1.1 MB binary (was a hard link error before).
list_installed_libraries() now honors VELXIO_FALLBACK_SKETCHBOOK (pro overlay) so
GET /api/libraries/list enumerates the content-addressed cache instead of the
shared global dir — the Installed tab survives the global volume's retirement
(audit finding #3). Unset (OSS self-host) -> default sketchbook, unchanged.
The last paths that read the shared global /root/Arduino/libraries: a compile
with NO manifest (libraries=null — any from-scratch/anon sketch; the manifest is
never auto-derived from #includes) and the incomplete-manifest scan-all retry
(which re-enters compile unscoped). Both fell through to the global dir,
bypassing the cache entirely (a cached lib still failed when global was gone).
Now, when no scope is materialized, point the library search at the cache: the
cache root is itself a valid Arduino libraries dir (each <name@ver-sha> child is
a library), exposed via env (pro overlay sets them):
- arduino-cli: ARDUINO_DIRECTORIES_USER = VELXIO_FALLBACK_SKETCHBOOK (whose
libraries/ -> cache root) when scope_dir is None.
- ESP-IDF: _find_arduino_libraries_dir() prefers VELXIO_FALLBACK_LIBRARIES_DIR.
Unset (OSS self-host) -> legacy global, unchanged.
Also strip a trailing @version from manifest names (_bare_lib_names): norm_name
fused 'ArduinoJson@6.21.5' -> 'arduinojson6215' (cache miss -> global scan-all);
the per-board boards_json manifest is the path that still carries @version.
The compile scope resolved a project OWNER's per-user custom libraries for ANY
project_id a requester supplied, with no visibility check — so a requester who
knew a victim's PRIVATE project_id + custom-lib name could compile a binary
against the victim's private uploaded library. Replace the ungated
get_project_owner hook with resolve_compile_owner(project_id, requester_id),
which returns the owner ONLY when the requester IS the owner OR the project is
shareable (public/unlisted); otherwise None -> the caller falls back to the
requester's OWN store. Fails closed on any error.
Also gate the server-side manifest fallback by the same rule (symmetry): a
private project's declared library NAMES are no longer read into a non-owner's
compile scope. The owner's own compile and shared/embed compiles of public/
unlisted projects keep their backend-authoritative scope unchanged.
Empirically, arduino-cli's --libraries ADDS to the search path — the global
sketchbook is STILL scanned, so the prior commit did NOT isolate reads from the
shared global volume. Point ARDUINO_DIRECTORIES_USER at the scratch sketchbook
instead (its <scratch>/libraries becomes the ONLY user-library dir); cores +
board-manager URLs live in the DATA dir and are untouched.
In-container functest (AVR uno): a cache-only lib compiles via the scope (never
in global); a global-only lib NOT in the manifest is INVISIBLE to the scoped
compile and only recovers via the scan-all retry (manifest_incomplete=True) —
proving the global volume is no longer scanned for a manifest-scoped build.
arduino-cli compiles now read libraries from a per-compile --libraries dir of
symlinks materialized by the pro overlay (owner store -> cache -> legacy), via
the existing materialize_library_scope hook, instead of scanning the shared
mutable global volume. --libraries only overrides the USER library search path,
so cores + board-manager URLs (RP2040 earlephilhower, ATTinyCore) are untouched.
Mirrors the ESP-IDF P2.1e graceful fallback: an incomplete manifest (a needed or
transitive lib not declared) makes the scoped compile miss a header; we retry
ONCE scan-all (no --libraries) and surface manifest_incomplete, so a partial
manifest degrades to legacy behavior instead of hard-failing.
Dedup correctness: the manifest + owner are resolved ONCE (shared
_resolve_compile_scope) and folded into the /compile/start dedup key AND threaded
into the build, so two owners with identical sketch+board but different custom
libs never coalesce to one another's job, and the key never diverges from the
bytes the build uses. Owner folded only when a manifest applies (index-only
compiles keep cross-owner dedup).
New OSS warm_library hook; /api/libraries/install now calls it (with the
requester id for the anon policy) instead of mutating the shared global
libraries volume — so that volume stops growing and can be retired (P2.5).
Falls back to the legacy arduino-cli global install when no overlay is loaded
(OSS self-host parity).
- compile.py: owner_id = project owner ELSE the requester (so an unsaved
compile resolves the libs the user just uploaded, which are their own);
threaded requester_id into _run_compile from both call sites.
- LibraryManagerModal: on a custom .zip upload, auto-add the lib to the active
board's velxio.json + show the Project tab, so the compile resolves it via the
owner per-user path (the upload now lands in the per-user store, not the
shared dir, so it must be declared to be found).
So a scoped compile can resolve the project OWNER's per-user custom libraries
(not the requester's) — a shared/embed/anon compile of someone else's project
still finds that owner's uploaded libs.
- core/hooks.py: new get_project_owner hook; materialize_library_scope gains an
opaque owner_id param (no-op default unchanged).
- espidf_compiler.compile/_attempt: thread owner_id to the materializer.
- compile.py: resolve owner via get_project_owner(project_id), pass to compile.
Additive: the OSS image (no overlay) ignores owner_id; index libs still resolve
from the cache. Foundation for per-user custom-lib storage (P2.2a write side).
A scoped ESP32 compile can now resolve libraries from a per-compile directory
provided by an overlay (the manifest's libs symlinked from a content-addressed
cache, with a legacy-dir fallback) instead of the single shared global volume.
- core/hooks.py: register_materialize_library_scope / materialize_library_scope
(no-op default -> None, so the OSS image keeps its single scan-all dir).
- espidf_compiler: _attempt(allowed) calls the hook, folds the returned content
token into the build-variant eff_hash (a content change gets a clean build
dir), passes libraries_dir to _compile_in_dir (arduino_libs = libraries_dir or
_find_arduino_libraries_dir()), and removes the throwaway dir after. The
graceful scan-all fallback (allowed=None) keeps using the default dir, so the
worst case of any materializer failure is fall-back-to-legacy (no break).
Library manifests are now PER-BOARD (each board carries its own velxio.json),
so two boards in one project can use different (even conflicting) libraries
without clashing — the multi-board extension of the no-clash guarantee.
- board.libraries on BoardInstance + serialisableBoard: rides in boards_json,
so it round-trips, dirty-checks, autosaves and restores natively. This also
removes the load-restore hacks (useLibraryManifestStore + applyProjectManifest
deleted): the manifest is plain board state.
- loadProjectState now restores per-board boardOptions/spiffsFiles/libraries
(it previously dropped them).
- EditorToolbar single + compile-all send the COMPILING board's libraries.
- Backend compile.py prefers the client's per-board request.libraries; the
project-level libraries_json (now the union of all boards) is the fallback.
- buildLoadPayload migrates pre-per-board projects: seed each board with the
project union so they keep compiling scoped.
- Library Manager 'In project' tab edits the ACTIVE board's velxio.json (shows
the board name) and the add field is now an autocomplete (installed libs +
index search) so users pick from a list instead of typing names.
Deletes useLibraryManifestStore.ts + applyProjectManifest.ts.
Adds the get_project_libraries hook: the compile route reads a saved project's
declared library manifest (by project_id) and uses it as the ESP-IDF resolution
scope, preferring it over the client-sent manifest. So a saved project always
compiles against only its own declared libraries — never another user's, or
another project's, stray install in the shared dir — authoritatively from the
server, independent of frontend wiring. Client-sent manifest still used for
unsaved examples; None/empty → legacy scan-all. Overlay fills the hook in
register_pro; OSS default is no-op (None).
Per-variant build dirs fixed the cross-compile staleness, but the FIRST build of
a cold variant can still occasionally hit ESP-IDF nested-build flakiness (cmake /
bootloader / managed-components / sdkconfig). These are infrastructure failures,
never user code, and clear on a retry once the variant dir is warmer. Retry the
attempt once when the failure matches infrastructure markers (not a user-sketch
or missing-library error). Cheap via ccache + ninja incremental.
Replaces the wipe-on-change approach (which left ESP-IDF's nested bootloader /
managed-components build in a broken state under rapid reconfigure -
intermittent 'managed_components_list.temp.cmake: No such file'). Each distinct
configuration (board options x resolved library set, via the variant key the
caller already computes) now gets its OWN persistent project dir with its OWN
build/, never wiped or reconfigured for a different config:
- same config -> same dir -> warm ninja incremental + ccache (fast iterate);
- different config -> different dir -> isolated, consistent, no staleness,
no nested-build breakage;
- the scoped vs scan-all fallback attempts land in different dirs, so the
double-compile no longer corrupts a shared build/.
Variant dirs are LRU-bounded (_MAX_BUILD_VARIANTS per target); the global ccache
warms a fresh/evicted variant in seconds. Cleans up the old single-project
layout on first run. Fixes the cross-compile staleness for legacy AND manifest
compiles, and the fallback regression.
Supersedes the mid-_compile_in_dir build/ wipe (161deb9): wiping build/ AFTER
materializing libs but right before cmake left ESP-IDF's config half-regenerated
during the fallback's scoped->scan-all double-compile, intermittently failing
with 'sdkconfig.h: No such file'.
Instead fold the effective library set (the manifest, else the sketch's
non-core external includes) into the per-attempt build-dir hash, so a changed
lib set — or the scan-all fallback after a scoped attempt — resets the
persistent build/ at _prepare_persistent_project_dir time (before any cmake).
That is the existing, well-tested early-wipe path, so the configure is always
clean. Same lib set across compiles keeps the warm ccache/ninja cache; core-only
sketches share one dir (core headers filtered out of the token) so they never
trigger a spurious wipe.
Fixes both the original cross-compile staleness (intermittent cmake-configure
failures + stale-object false positives) and the fallback regression.
The persistent per-target build/ caches ESP-IDF's cmake configuration, ninja's
incremental graph and ccache-backed objects, all assuming a stable component
set. When consecutive compiles on the same dir have a DIFFERENT resolved
user_libs set (a different project/user, or a different library manifest) that
cache is inconsistent and produces two real failures:
- cmake reconfigure intermittently fails ('cmake configure failed') even
though each manifest compiles fine on a clean dir;
- ninja/ccache reuse a previous compile's objects/headers, letting a
now-absent library slip through as a false-positive success against a lib
the current sketch/manifest no longer includes.
Fingerprint the materialized user_libs/ (sorted relative paths + sizes) and,
when it changes vs the last compile on this dir, wipe build/ to force a clean
configure. ccache (enabled) refills the objects so the rebuild stays cheap.
No-op on the ephemeral path and the first compile. Fixes both symptoms; will be
superseded by the fully ephemeral per-compile workspace (P1).
A manifest-scoped compile that fails because a header isn't in the manifest
(an undeclared/transitive dependency) now retries once with scan-all, so a
project with an incomplete manifest still compiles instead of regressing.
The response reports manifest_incomplete=true and
manifest_suggested_libraries={header: [candidate lib names]} so the manifest
can be auto-completed (P2.4) or the user prompted to add the missing library.
This de-risks turning on manifest sending (P2.3): an incomplete example/project
manifest can never break a build that worked before.
- compile(): _attempt(allowed) helper; retry scan-all on missing-lib failure.
- _missing_library_headers / _suggest_libraries_for_headers helpers.
- CompileResponse.manifest_incomplete + manifest_suggested_libraries.
P2.0 first cut took the first-alphabetical lib providing a header and then
checked manifest membership. When several installed libs ship the same header
(e.g. DHT118266, DHT_sensor_library, servodht11 all have DHT.h), the stray
first-match got rejected and the header was dropped even though the declared
lib provides it.
_find_manifest_library_for_header: when a manifest is supplied, pick the first
DECLARED library that provides the header. This both selects the right lib and
excludes undeclared ones. No manifest = legacy first-match.
Test strengthened with a stray same-header lib that sorts first.
When a compile supplies a 'libraries' manifest, _resolve_library_components
merges a USER-installed library only if it's declared in that set. A sketch
therefore never picks up an unrelated library from the shared dir (another
user's install, or a same-named clash) — the manifest is the resolution scope.
- _resolve_library_components(allowed_libraries): gate user-lib merges on
manifest membership; match by folder name OR library.properties name=,
normalised (display name vs on-disk folder differ by separators/case).
Core/bundled libs are never gated. None = legacy scan-all (unchanged).
- Threaded through compile() -> _compile_in_dir.
- compile.py: CompileRequest.libraries; folded into the async dedup _job_key
so a different manifest doesn't dedup to a job built with another.
Opt-in: omitting 'libraries' preserves current behaviour exactly.
Regression: test/backend/unit/test_espidf_core_first.py::TestManifestScope
A user library that ships a core-named header (e.g. WiFiEspAT/src/WiFi.h)
could shadow the arduino-esp32 core during ESP-IDF library resolution.
WiFiEspAT shadowing WiFi.h pulled EspAtDrv.cpp into the build, whose
const char OK[]/STATUS[] collide with ESP-IDF's enum STATUS in
rom/ets_sys.h, breaking every ESP32 sketch that #include <WiFi.h>.
_resolve_library_components now:
- skips a header entirely when the arduino-esp32 core provides it
(computed set from cores/ + libraries/, cached), so a user lib can
never shadow WiFi.h/Wire.h/SPI.h/WebServer.h/...
- skips a resolved user lib whose library.properties architectures=
excludes esp32/*.
Regression: test/backend/unit/test_espidf_core_first.py
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The 7.5" 800x480 dashboard (GxEPD2_750_T7) rendered blank: it is a UC8179 /
GD7965 controller, but the panel config claimed controllerFamily 'ssd168x',
so the SSD168x decoder (which only reads 0x24/0x26/0x44/0x45) ignored its
0x10/0x13 DTM stream.
- Add a Uc8179 decoder (worker Uc8179EpaperSlave + browser Uc8179Decoder).
UC8179 is the same UltraChip command family as the UC8159c (0x10/0x13 DTM,
0x12 refresh) but mono (1 bit/px). GxEPD2 writes the visible image to 0x13
(DTM2 "current"; 0x10 is the ignored "previous"), framed by 0x91/0x90
(partial window, pixel coords MSB-first)/0x13 data/0x92. Data lands at
absolute pixel coords inside the window, so compose is just the RAM. The
Frame reuses the SSD168x palette (0=black, 1=white) so paintFrame renders it.
- EPaperPanels.ts: add the 'uc8179' family and point epaper-7in5-bw at it.
EPaperPart.ts + esp32_worker.py dispatch 'uc8179' to the new decoder.
- Fix the BUSY polarity: UC8179 (like the UC8159c) idles BUSY HIGH, not LOW.
The worker seeded BUSY LOW for every non-uc8159c panel, so GxEPD2_750_T7's
_PowerOn()/_InitDisplay() busy-wait timed out (~10 s, "Busy Timeout!") on
every refresh. Now _PowerOn returns in ~129 us.
- esp32_worker.py: the runtime sensor_attach epaper path still emitted the
epaper_update payload nested under 'data' (the old double-wrap bug); emit
it flat like the init path.
The 5.65" ACeP UC8159c example already rendered (it has its own decoder and
got the WS-plumbing fix); verified the 7 colour bars are correct.
The 2.9" tri-colour ESP32 alert badge rendered the red ALERT pill as white:
the red plane (0x26) was received but landed out of bounds and was dropped.
GxEPD2_3C writes the 0x24 (black) plane then the 0x26 (red) plane WITHOUT
re-seeking the RAM address counter between them — it relies on the SSD168x
counter wrapping back to the window start after the last byte of the window.
Our decoder advanced Y past the window end instead of wrapping, so every
0x26 byte hit y >= rows and was discarded (red_ram stayed all-init).
Mirror the hardware: when the X cursor wraps at the end of a row, advance Y
with a wrap at the active window boundary (yrange), honouring the data-entry
Y direction. Applied identically to the worker slave, the browser decoder,
and the Python golden reference so the three stay in lockstep. No regression
on the mono panels (their counter is re-seeked per plane, so the wrap is a
no-op for them); verified the tri-colour pill now renders red and the 2.9"
weather / 2.13" clock / 1.54" hello panels are unchanged.
ePaper panels rendered rotated/misaligned on AVR and RP2040 (e.g. the 2.13"
Pico clock came out sideways and clipped). The ESP32 worker decoder was just
taught to compose in the controller's native RAM geometry and rotate to the
display orientation, but the browser-side SSD168xDecoder (used by AVR/RP2040)
still composed at display dims with no rotation, so the two diverged.
- SSD168xDecoder.ts: port the worker's native-window compose + rotation.
* Size RAM to the longer side both ways so a rotated native layout
(128x296 behind a 296x128 panel) isn't truncated.
* Compose in the active RAM window, then rotate via the inverse of
Adafruit_GFX setRotation(1). Detect orientation by BYTE width so a
non-multiple-of-8 native width (the 2.13" panel is 122 px) is handled.
* Track the UNION of windows per frame: paged drivers (GxEPD2 page height
< panel) set one partial window per page, so compose must use the full
native area, not just the last page's strip. Fixes the all-white render
on paged panels (1.54" Uno, 4.2" Pico, 7.5" ESP32).
* Add an isBwr option: B/W panels treat 0x26 as a 2nd mono plane (white
only if both planes white), tri-colour panels keep red-wins.
* Default the active window to display geometry; the firmware overrides it.
- EPaperPart.ts: pass isBwr = cfg.palette === 'bwr' to the decoder.
- esp32_spi_slaves.py / esp32_worker.py: mirror the byte-aware rotation +
window-union in the worker, and derive is_bwr from panel_kind on the
runtime sensor_attach path too (fixes the tri-colour ESP32 alert badge).
- test_epaper/ssd168x_decoder.py: re-port the golden reference to match
(keeps the 3-way TS/Python/worker identity invariant). Tests updated to
construct tri-colour cases with is_bwr/palette='bwr'.
- examples-displays-epaper.ts: the Pico VCC wire referenced '3V3(OUT)',
which the velxio-pi-pico-w element doesn't expose (it has '3V3'), so the
wire snapped to the board corner. Use '3V3'.
ESP32 ePaper examples (e.g. epaper-2in9-esp32-weather) rendered as a blank
white panel. Two bugs, both above the SPI layer:
1. The worker's epaper_update event nested its payload under 'data', unlike
every other (flat) worker event. The backend qemu_callback re-wraps the
post-'type' payload under 'data', so the frontend received
msg.data.data.component_id (undefined) and EPaperPart bailed on
id !== componentId, so paintFrame/putImageData never ran. Emit it flat.
2. Ssd168xEpaperSlave was sized to the display dims (296x128), but GxEPD2
with setRotation(1) writes the controller's NATIVE RAM (128x296). The
_y < height bound dropped rows 128-295 (half the image) and compose
never rotated. Size RAM to the longer side, compose in the native
active-window geometry (0x44/0x45), then rotate to the display
orientation (inverse of Adafruit_GFX rotation 1). Add is_bwr (from
panel_kind): B/W panels init the 0x26 plane white and compose
white-only-if-both (GDEY029T94 mirrors the image into 0x26); tri-colour
panels keep red init 0x00 and red-wins.
Worker side of the libqemu picsimlab_spi_event_batch / CS-gating change:
- _on_spi_batch(): replay a whole SPI transfer in bulk (custom-chip runtime,
then ePaper feed, then the spi_batch buffer) instead of one _on_spi_event per
byte. Registered as a trailing _SPI_BATCH field of _CallbacksT.
- _sync_cs_events(): disable SPI chip-select callbacks for pure-display sims,
enable them when an ePaper / custom-chip SPI slave is registered (no-op on
older libqemu without qemu_picsimlab_enable_spi_cs_events).
- _on_pin_change(): flush the SPI batch before each gpio_change so the byte
stream stays ordered against the DC pin now that CS no longer triggers the
flush.
Backward compatible: an older libqemu never calls the batch callback or the CS
setter, so it just keeps the per-byte path. esp32-doom: 0.04 -> ~1.0-1.5 FPS
wall-clock (~26-37x), render verified correct.
SDCC's z80 crt0 puts the reset vector at 0x0000 (jp init) and the init stub
(set SP, call _main) at an absolute .org 0x100. Passing `--code-loc 0x100`
relocated the _CODE segment on top of that init stub, so on reset the CPU
jumped into __clock/_exit (rst 0x08 then ret with a garbage stack) and
derailed into NOP land before ever reaching main — every Z80 C program ran
but drove nothing (z80-led-chaser-c compiled yet the LEDs never moved).
Verified via a standalone chip-WASM harness: with the flag the chaser does 0
LED writes; without it, it walks the bit (8 writes). Pairs with the z80-cpu
RAM map now covering 0x8000-0xFFFF so the crt0's SP=0 stack is real RAM.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds stm32-f4-discovery, stm32-olimex-h405, stm32-netduino-plus2, stm32-netduino2, stm32-blackpill-f401 and stm32-bluepill-f103cb, mapped to existing qemu-lcgamboa machines (netduinoplus2, olimex-stm32-h405, netduino2, stm32vldiscovery). A generic inline board renderer (no SVG) draws the Discovery/Olimex/Netduino boards from a header pin layout; the Pill variants reuse the Blue/Black Pill SVGs. Per-board onboard-LED pin and polarity via STM32_LED. One blink+serial example per board.
tsc --noEmit clean; all new FQBN pnum variants present in STM32 core 2.12.0; worker smoke tests pass for the new machines.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
STM32 emulation (open-core, runs via libqemu-arm in the backend worker):
- backend: stm32_lib_manager + stm32_worker (GPIO, USART, I2C/SPI device models
reusing the ESP32 slaves, live sensor updates), arduino_cli STM32 branch,
start_stm32 simulation route.
- frontend: Stm32Bridge + Stm32BluePill(/BlackPill) web components (Wokwi SVGs),
board kinds, Interconnect/boardPinMapping/boardProtocols wiring, example
projects (blink, serial, I2C BMP280/MPU6050/DS1307/SSD1306/weather, 7-seg,
RGB, button, switch, stepper, cross-board interconnect).
- Raspberry Pi 4/5 board elements + thumbnails.
Pro board gating (generic OSS->Pro seam; entitlement logic lives in the overlay):
- lib/proBoardGate.ts: isProBoardKind (STM32 + every QEMU Raspberry Pi),
installBoardGateImpl/boardGateDecision, triggerProUpgradePrompt.
- PRO badge on those boards in the component picker; gate at the picker add +
the run backstop (startBoard).
- backend/app/services/board_access.py: server-side enforcement seam for the
simulation WebSocket; STM32/Pi unavailable -> Pro-framed message.
- desktop: generic QemuDownloadPrompt + Stm32QemuPrompt (download-behind-license,
mirrors the ESP32 prompt).
- .gitignore: never ship libqemu-* binaries in the public image.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two unrelated polish fixes.
espidf_compiler: headers that resolve to an arduino-esp32 CORE lib
(WebServer, WiFi, …) were correctly skipped from the user-lib merge but
then fell through to a scary "Library for <X> not found — build may
fail" warning — even though the build succeeds because the symbols are
compiled into the core. Now logs an accurate "provided by arduino-esp32
core — already compiled in, not merging". Same treatment for core
headers that aren't standalone lib dirs (Udp.h, IPAddress.h,
WiFiUdp.h, …) via a new _CORE_ESP32_HEADERS allowlist.
SimulatorCanvas: the WiFi badge's "open IoT gateway" click now consults
an optional window.__velxio_iot_gateway_open_gate__ hook before opening
the gateway tab. A private overlay can install it to gate the gateway
behind a paid plan and show an in-place upgrade modal instead of dumping
a 402 page in a new tab. OSS builds have no hook → opens normally. The
check is synchronous so it doesn't trip popup blockers.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a generic gating hook so a private overlay can restrict the IoT
gateway proxy to paid plans without the OSS image carrying any plan
logic. register_iot_gateway_gate() installs an async callback that
returns None to allow or a detail dict to block; the OSS default (no
overlay) allows everyone, and a failing gate fails OPEN so the gateway
can never be taken down by a buggy overlay.
gateway_proxy() calls the gate first. When blocked it content-
negotiates the 402: browsers (Accept: text/html — the frontend opens
the gateway via window.open) get a small styled upgrade page with a
link to /pricing; programmatic fetch/XHR callers get the JSON detail.
No behaviour change for the open-source image — the gate is a no-op
there.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Brings hardware flashing into Velxio Desktop. Per-board "Flash to
real board" entry in the canvas context menu opens a modal that
enumerates USB serial ports, lets the user pick one, then
streams arduino-cli upload output live until the board is flashed.
Backend (Phase D1) — backend/app/api/routes/flash.py (new):
POST /api/flash/upload (multipart: board_id, port, fqbn,
program_format, program)
→ SSE stream of {phase, line?, progress?} events
→ final {phase:'done', success, elapsed_ms, error?}
- Wraps `arduino-cli upload -p <port> -i <file> --fqbn <fqbn> -v`
so AVR (avrdude), ESP32 (esptool), RP2040 (picotool), SAMD
(bossac) all share one code path — arduino-cli internally
dispatches by FQBN.
- Per-port asyncio.Lock prevents two simultaneous flashes from
fighting over the same /dev/ttyACM0.
- Allow-list of FQBN prefixes (arduino:avr, ATTinyCore:avr,
rp2040:rp2040, esp32:esp32, arduino:samd) so a typo can't
cause a confusing arduino-cli error.
- Format allow-list (hex / bin / uf2 / elf) drives the temp
file extension - arduino-cli uses the extension to route to
the right uploader.
- 8MB hard cap on the uploaded program (real sketches are
well under that; protects against a runaway frontend).
- X-Accel-Buffering: no header so nginx doesn't hold the SSE
chunks until the flash completes.
Frontend (Phase D3):
- frontend/src/services/flashService.ts (new):
async generator streamFlash() yields parsed SSE events.
Handles the base64-vs-text gotcha (compile returns hex_content
as text but binary_content as base64; for binary formats we
atob() into a Uint8Array before posting so the form upload
sends actual bytes, not the base64 ASCII).
- frontend/src/components/simulator/FlashModal.tsx (new):
Three-state UI: picking (port dropdown), flashing (progress
bar + live log), success/error (verdict + retry).
Empty-ports state shows a Linux dialout-group hint.
- SimulatorCanvas.tsx: board context menu gains "Flash to real
board" entry, gated on isTauri() + presence of compiledProgram.
Hidden in web (WebSerial is a separate sprint).
- tauriBridge.ts: SerialPortInfo type + listSerialPorts() helper
that invokes the Rust shell command added in Phase D2.
The sidecar already has arduino-cli on PATH (per
`pro/desktop/sidecar/main.py::_expose_bundled_arduino_cli`), so
no installer changes are needed — flash works the moment the
0.4.x desktop bundle ships with these commits.
Plan + remaining phase tracked in project/hardware-flashing/.
D2 (Rust serial enum) committed separately as a Tauri-shell-only
concern; D4 (manual smoke matrix with real boards) requires
physical hardware so it stays a follow-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The desktop Tauri wrapper drops the downloaded libqemu-xtensa.<ext>
inside the directory it exports as VELXIO_QEMU_PATH. The sidecar
previously only checked the dedicated QEMU_ESP32_LIB env var (full
path) plus a fixed file beside the module, so the in-app installer's
output was invisible — adding an ESP32 to the canvas right after
install reported "unavailable" until the sidecar was restarted.
Two changes:
1. New _resolve_lib() helper checks QEMU_ESP32_LIB first, then
VELXIO_QEMU_PATH/<lib_name>, then the legacy module-adjacent path.
Both Xtensa and RISC-V flow through the same resolver.
2. lib_xtensa_path() / lib_riscv_path() functions replace the module
constants for all internal call sites (is_available,
is_riscv_available, start_instance). Resolution happens per call,
so a post-boot install is picked up without restarting the
sidecar. The LIB_PATH / LIB_RISCV_PATH constants stay for any
external readers but reflect import-time state only.
Pairs with the Tauri-side change that writes libqemu-xtensa.<ext>
directly to VELXIO_QEMU_PATH (no archive extraction).
User report after the v0.4.0 desktop agent landed:
> agente devuelve "LLM call failed: Failed to fetch"
"Failed to fetch" is a network-layer error, not 401. Root cause: the
OSS CORS allow_origins list only included http://localhost:517[3-5]
(vite dev) and settings.FRONTEND_URL. The desktop bundle runs from
either tauri://localhost (macOS/Linux) or http://tauri.localhost
(Windows) - both cross-origin to velxio.dev - so the browser
blocked the agent's POST /api/pro/agent/llm preflight before the
backend ever saw it.
Added all three Tauri scheme variants to the allow list. After this
lands + a backend restart the desktop agent's fetch reaches the
real /api/pro/agent/llm and the dual-auth dep from v0.4.0 Phase 1
gets to do its job (Bearer license-key → resolved User → quota
check → upstream LLM proxy).
Origins added:
tauri://localhost # macOS / Linux (Tauri 2.x default)
http://tauri.localhost # Windows (Tauri 2.x default)
https://tauri.localhost # older Tauri 2.x releases
allow_credentials stays True - the existing cookies-from-web flow
still works, the Tauri origins just don't have any cookies to send.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A 1024-byte oled.show() writevto generates ~1025 calls into
_on_i2c_event, one per byte. The previous code called _log (stderr
write+flush) AND _emit (stdout JSON write+flush) for every event,
saturating the worker subprocess's stdout pipe. The QEMU thread blocks
on the synchronous write, the firmware's ESP-IDF i2c_master ISR
re-enters before the previous one finished, and the Interrupt watchdog
trips on CPU1 with a "Guru Meditation Error: Interrupt wdt timeout"
panic on the second consecutive oled.show() call.
I2CWriteSink already buffers writes internally and emits a single
i2c_transaction event on FINISH, so the per-byte log+emit was pure
overhead with no observability value for display drivers (SSD1306,
PCF8574). Keep them for everything else.
Verified end-to-end via chrome devtools MCP:
- Test minimal (init + 1 explicit show): markers OK, no panic
- Loop test (10x oled.show() + sleep(0.5)): DONE_LOOP, no panic
ESP-IDF runs component_get_requirements.cmake in script mode (not full
project mode) to scan component dependencies before the build proper.
CMake rejects CONFIGURE_DEPENDS in script mode with:
CONFIGURE_DEPENDS is invalid for script and find package modes.
The persistent build dir is already wiped + restored from the template
on every compile (espidf_compiler.py:220-222 + CMakeLists.txt patches),
so the glob is re-evaluated naturally on the next cmake configure — we
don't need CONFIGURE_DEPENDS to catch new files.
Pairs with 27c9a28 (glob fix) + 21791a8 (template main.cpp decouple).
Now that main/CMakeLists.txt globs every *.cpp/*.c into SRCS (commit
27c9a28 — needed so multi-file Arduino projects link), sketch.ino.cpp
is compiled as its own translation unit. The template main.cpp's
`#include "sketch.ino.cpp"` is left over from when SRCS only named
main.cpp — back then we had to pull the user code into main.cpp's TU
to get it compiled at all.
With the glob in place, keeping the include re-defines setup() and
loop() in main.cpp's TU AND in sketch.ino.cpp.obj, so the linker dies
with "multiple definition of `setup'". Replace the include with a
forward declaration and let the linker resolve setup()/loop() from
sketch.ino.cpp.obj. Matches arduino-cli's per-file compile + auto-link
model, and frees pure-C++ multi-file sketches to put helper TUs in
their own .cpp files without textual-include hacks.
Repro: robot-desktop-eyes after the glob fix (#13 rebuild) — compile
got past resolve+compile and into link, where multi-def of setup()
would have appeared on the next attempt. This commit closes the loop.
Multi-file Arduino sketches (robot-desktop-eyes is the canonical
case) define classes and free functions in their own translation
units — Face::Update(), FaceExpression::GoTo_Surprised(),
AsyncTimer::SetIntervalMillis(...), etc. live in Face.cpp,
FaceExpression.cpp, AsyncTimer.cpp respectively.
The espidf_compiler drops every user-supplied .h/.cpp into
project/main/ but the main/CMakeLists.txt only declared
`SRCS "main.cpp"`, so the helper TUs never got compiled and the
linker died with dozens of "undefined reference to ..." entries.
Switch SRCS to a CONFIGURE_DEPENDS glob over the main/ directory
so every .cpp/.c that lands in main/ becomes part of the IDF main
component automatically. CONFIGURE_DEPENDS makes CMake re-evaluate
the glob on every reconfigure, which matters because the persistent
build dir is reused across compiles and the set of helper files
changes per sketch.
Repro: open robot-desktop-eyes, click Compile. Before this commit:
37 linker errors starting at "undefined reference to `u8g2'" and
"undefined reference to `Face::Update()`". After: the helper TUs
compile and the .elf links.
Two related fixes for the ESP32 Arduino-compat compile path:
(a) backend/app/services/esp-idf-template/main/CMakeLists.txt:
Demote -Werror=comment / =parentheses / =sign-compare / =narrowing
/ =write-strings / =missing-field-initializers / =reorder back to
plain warnings. ESP-IDF's project defaults are stricter than what
Arduino/arduino-cli users expect, so common Arduino idioms (nested
/* */, missing field initializers in struct literals, etc.) were
failing builds that compile fine in the Arduino IDE. -Wall stays
on; we just stop the abort.
(b) examples-robot-desktop.ts (robot-desktop-eyes example):
Replace the nested /* xTaskCreatePinnedToCore( ... /* Task function. */
... */ block with `#if 0 / #endif` so the inner block comments
don't terminate the outer one. Even with -Wno-error=comment the
real-syntax-level issue (the first inner `*/` closes the outer
comment, leaving the rest of the lines as bare code) would still
bite, so this needs an actual code fix.
When a sketch's external library headers were only referenced from
project headers (e.g. esp32-eyes.ino includes Common.h, Common.h
includes <ESP32Servo.h>), the compile failed with
fatal error: ESP32Servo.h: No such file or directory
because _detect_external_includes was only called on main_content
(the processed .ino). Project .h/.cpp files were never scanned, so
ESP32Servo / DHT / Adafruit_Sensor referenced only transitively
through user code never reached _resolve_library_components and
never landed in user_libs_all/.
Fix: collect ext_headers from main_content PLUS every uploaded
.h/.hpp/.ino/.c/.cpp file before resolving libraries. Lib resolver
already walks transitive includes inside the lib bundle once it's
copied; this just makes sure the first-level set covers user
project headers too.
Repro: open https://velxio.dev/example/robot-desktop-eyes, click
Compile. Before this commit: 13 errors starting at ESP32Servo.h.
After: ext_headers includes ESP32Servo.h on the first pass and the
build proceeds.
Two intertwined bugs were leaving every ESP32 ePaper example broken
end-to-end. Only the 5.65" UC8159c panel surfaced the failure
audibly ("Busy Timeout!" repeating in serial), because its inverted
busy polarity caused the firmware to hang inside `_waitBusy()`. The
SSD168x ePaper examples APPEARED to run cleanly but never actually
rendered anything to the panel — the canvas stayed at the idle paper
colour because the same registration path was broken.
Root cause #1 — `setSensors` was a full REPLACE, not a merge.
`Esp32Bridge.setSensors(sensors)` did `this._pendingSensors =
sensors`. At `startBoard()` time the store iterates components,
resolves wires for any entry in `SENSOR_COMPONENT_MAP` (DHT22 /
HC-SR04 / I²C sensors) and calls `setSensors(...)` with that list.
ePaper components live in `PartSimulationRegistry` (not in the
sensor map) and are registered via `sendSensorAttach()` AT
COMPONENT-MOUNT TIME — well before `startBoard()` runs. Full-replace
semantics blew that registration away on every Run click, so the
worker never instantiated an `Ssd168xEpaperSlave` / `Uc8159cEpaperSlave`,
no SPI bytes were decoded, no frames were latched, and BUSY was
never driven.
Fix: upsert by `pin` so pre-existing registrations from
PartSimulationRegistry handlers are preserved alongside the
startBoard-resolved sensors. Confirmed via a WebSocket spy that the
`start_esp32` payload now carries the ePaper sensor entry.
Root cause #2 — BUSY polarity was hard-coded for SSD168x only.
Verified against upstream GxEPD2 source:
* SSD168x family — constructor passes `_busy_level = HIGH`
→ BUSY=HIGH means busy, LOW means ready.
* UC8159c family — constructor passes `_busy_level = LOW`
→ BUSY=LOW means busy, HIGH means ready.
The worker only drove BUSY after a frame flush (and at the wrong
polarity for UC8159c), so the firmware's first `_waitBusy()` inside
`_PowerOn()` / `_InitDisplay()` — which fires BEFORE any frame —
blocked for the full 25 s `_busy_timeout`.
Fix: read `controller_family` from the registration payload, pick the
per-family idle level, and (a) seed the pin to IDLE at registration so
the first `_waitBusy()` sees "ready" immediately, (b) use that
polarity (idle vs. busy) when pulsing on frame flush.
Verified on https://velxio.dev/example/epaper-5in65-7c-esp32-rainbow:
the serial timeline now reads `_InitDisplay reset : 1566` /
`_PowerOn : 148` / `_PowerOff : 183` / `frame done` (all sub-2 ms
busy-waits, no timeouts). Sensor registration confirmed via the
`start_esp32` payload carrying the `epaper-ssd168x` entry.
The user_libs_all bundler in _resolve_library_components does BFS over the
sketch's external includes, copying each matching Arduino library into
one merged IDF component. Anything not in _BUILTIN_HEADERS is treated as
an external library to resolve, and the lookup just scans
/root/Arduino/libraries/ for a directory whose `src/` (or root) holds a
matching header file.
_BUILTIN_HEADERS only listed C headers (stdint.h, stdio.h, …). The C++
wrappers (cstdint, cstdio, cmath, …) and the STL containers (vector,
complex, string, …) were absent. Result: any library transitively
#including <cstdint> or <vector> caused the bundler to "resolve" the
header against /root/Arduino/libraries/ArduinoSTL/ — an AVR-only
uClibc++ port that ships every C++ stdlib header as plain files. Once
ArduinoSTL was matched the bundler dragged in ALL of it, including
complex.cpp:
template class _UCXXEXPORT complex<float>;
which fails on the ESP-IDF Xtensa toolchain because _UCXXEXPORT isn't
defined in that compile context AND the symbol already exists in the
real libstdc++ pulled in by <complex>. Net effect: every ESP32 sketch
whose deps transitively include a C++ stdlib header (e.g. ESP32Servo
includes <cstdint>) blew up with 66+ errors before the servo example
even reached the link step.
Fix: extend _BUILTIN_HEADERS to cover the full set of C++ stdlib
wrappers and STL headers so the bundler never treats them as installable
libraries. The Xtensa GCC + libstdc++ shipped by ESP-IDF provides them
natively; ArduinoSTL never has any business being part of an ESP32 build.
Verified end-to-end on /example/esp32-servo: compile now succeeds, sketch
boots, moving the potentiometer drives the wokwi-servo angle (Pot=2801 →
Angle=123 deg, servo arm rotates).
arduino-esp32 3.x ledcWrite takes a PIN and looks up the attached channel
internally. arduino-esp32 2.x (the toolchain version we pin) takes a
CHANNEL. The velxio_compat.h shim already aliased the 3.x-only
ledcAttach onto ledcSetup+ledcAttachPin so 3.x sketches would compile,
but ledcWrite still mapped 1:1 — so a call like
#define R_PIN 16
ledcAttach(R_PIN, 5000, 8); // shim → channel 0 attached to pin 16
ledcWrite(R_PIN, 128); // ★ writes to "channel 16" (invalid)
silently wrote to LEDC channel 16, which doesn't exist (valid range
0-15). The hardware duty register never changed, qemu-lcgamboa never
emitted a `ledc_duty` event, and the RGB LED stayed dark even though
the firmware ran cleanly and the wires looked right. Verified end-to-end
with examples/esp32-pwm-led-rgb: gpio_change events fired at boot, no
ledc_duty events fired, ledRed/ledGreen/ledBlue all stayed at 0.
Fix: maintain a 40-entry pin→channel table populated by both ledcAttach
variants. Replace ledcWrite with a macro that calls a helper checking
the table first; if the value isn't a known pin we pass it through as a
channel, preserving 2.x channel-style call sites.
Macro/function name collision is sidestepped with the standard
parenthesizing trick — `(ledcWrite)(channel, duty)` doesn't expand the
function-like macro because the token isn't followed by `(`.
Verified live on velxio.dev/example/esp32-pwm-led-rgb after hot-copying
the new header into the velxio-app container: ledRed/ledGreen/ledBlue
now cycle through the full HSV wheel as expected (samples: (255,41,0),
(41,255,0), (0,41,255), (232,255,0), …).
Single-file sketch only — the table is `static` (internal linkage) and
ledcAttach + ledcWrite live in the header. Multi-file sketches that
attach in file A and write in file B would each see their own table.
Acceptable for now since arduino-esp32 sketches are nearly always
single-file; revisit when we bump the toolchain to 3.x and can drop the
shim entirely.
ATTinyCore >=1.5.0 declares ATTinyCore:micronucleus@2.5-azd1b as a tool
dependency, hosted at https://azduino.com/bin/micronucleus/. That host
has been unreachable (connection refused) for extended periods, causing
every ATtiny85 compile to fail at the core-install step with:
Download failed: performing HEAD request: ... dial tcp ...: connection refused
Failed to install required core: ATTinyCore:avr
micronucleus is only used for USB upload — never for compilation — but
arduino-cli refuses to install a core whose tool deps cannot fetch.
Pin to 1.4.1, the last release whose micronucleus binary is hosted on
github.com (digistump release, reachable). The FQBN clock options we
ship (clock=16pll on attinyx5, etc.) are unchanged across 1.4.x.
- backend/app/services/arduino_cli.py: new CORE_INSTALL_VERSIONS map
consulted by ensure_core_for_board so the runtime auto-install
passes "ATTinyCore:avr@1.4.1" instead of unversioned latest.
- backend/Dockerfile and docker/entrypoint.sh: same pin so a fresh
image bakes 1.4.1 in and never hits the runtime fallback path.
Existing regression tests in test/backend/unit/test_arduino_cli_attinycore.py
still pass (they assert presence, not version).
Two CI failures landed after PR #196 (esp32-gpio-matrix-cb-callback)
merged. Both are independent and fixed here together.
1) **Backend E2E: ESP32 hangs at bootloader handoff.**
PR #196 added picsimlab_gpio_matrix_cb which fires on QEMU's
iothread. The handler did `_emit({...})` for every routing
change — and the ESP-IDF bootloader writes to gpio_out_sel
*hundreds* of times during early boot (each peripheral init
configures its matrix slot). Each emit acquires _stdout_lock
and writes to the worker→manager pipe. If the manager drains
even briefly slow, the pipe fills, write blocks, and the
iothread stalls — symptom: ESP32 reports `entry 0x400805e4`
then no Arduino setup() output for 75 s.
Fix: the iothread callback now ONLY mutates the SignalRouter
snapshot. It never emits. The 10 Hz poll thread
(_refresh_signal_routing) stays as the sole emitter, so the
wire-format event stream is unchanged. Benefit of having the
callback over poll-only is reduced worst-case routing-emit
latency (next poll tick vs up to 100 ms) and a warmer
snapshot dict for cheaper poll diffs.
2) **Frontend Tests: Node OOM at end of suite.**
117 test files run in one forks-pool worker. Several lazy-load
the ngspice emscripten module (~30 MB), the MixedModeScheduler
singleton, and other heavy modules whose dispose hooks aren't
reached because singletons leak across files. Cumulative heap
pressure exceeds Node's 4 GB default; the worker hits "Ineffective
mark-compacts near heap limit" AFTER all 1881 tests pass and
the OOM kill is reported by vitest as "Worker exited unexpectedly
/ Timeout terminating forks worker". This is not a real test
failure — every individual test passes.
Quick fix: pass NODE_OPTIONS=--max-old-space-size=8192 to the
`npm test` step. Long-term, the singletons should add dispose
hooks that test fixtures call in afterAll(), or the suite
should shard into multiple `vitest run --shard` invocations.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires the new synchronous GPIO Matrix callback exposed by
libqemu-{xtensa,riscv32} 1.1.0 (lcgamboa/qemu commit e178ff5).
Whenever the firmware writes GPIO_FUNCx_OUT_SEL_CFG_REG, the C
plugin now fires picsimlab_gpio_matrix_cb(gpio, signal_id) inline.
The handler:
- Treats signal_id == 0x100 or 0 as "matrix routing cleared" and
emits gpio_routing_clear.
- For LEDC HS/LS range signals (the only ones the frontend
SignalRouter currently consumes), updates the mirror and emits
gpio_routing.
- Drops other signals — the mirror does not need to track them
yet, and emitting them would only fatten WS frames.
Backwards compat:
- Older libqemu (<1.1.0) doesn't expose the new field; the
picsimlab_gpio_matrix_cb placeholder runs (no-op) and the
100 ms _refresh_signal_routing() poll thread continues to feed
the mirror. WS event shape is identical either way.
Burn-in: keeping the poll thread active in parallel with the
callback for now. Once telemetry confirms parity (per phase 4 doc
in velxio-prod/project/esp32-gpio-matrix-cb/), the poll thread
gets retired in a follow-up commit.
Adds a third format to /api/compile-rom: `c` (C source compiled by SDCC
to Z80 bytes). Same chip-program flow as 8080/Z80 asm — write C in a
project file, click Compile, click Run.
Backend:
- backend/app/services/c_compile.py — async SDCC wrapper. Locates the
sdcc binary on PATH (or via SDCC env var, or common Windows install
paths) and shells out with target=mz80 + --code-loc 0x100 --data-loc
0x8000. Parses the resulting Intel HEX into raw ROM bytes. Pure 8080
is rejected with a clear error (SDCC has no 8080 backend; Z80 ROMs
also run on the i8080-cpu chip if you avoid Z80-only ops).
- rom_compile.py: compile_rom is now async; the new c branch delegates
to c_compile. compile_rom_endpoint awaits it.
Frontend:
- romCompileService: RomFormat gains 'c'; formatForFile maps .c/.cpp to
'c'. isChipProgramFile intentionally still excludes .c — disambiguation
happens at the EditorToolbar level.
- EditorToolbar: the chip-program path also fires when a custom-chip
has programFile === activeFile.name (regardless of extension). That
lets .c files route to /api/compile-rom (SDCC) when bound to a CPU
chip, while .c files NOT bound to any chip continue to route to
arduino-cli as before.
Docker:
- Dockerfile.standalone adds `sdcc` to the apt-get install list, so the
prod image ships with SDCC out of the box.
Example:
- /examples/z80-led-chaser-c — z80-cpu chip + chaser.c (a Larson
scanner written in C with __at() MMIO definitions). Compiles cleanly
with SDCC's --code-loc 0x100 default crt0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
David Montero