Two unrelated polish fixes.
espidf_compiler: headers that resolve to an arduino-esp32 CORE lib
(WebServer, WiFi, …) were correctly skipped from the user-lib merge but
then fell through to a scary "Library for <X> not found — build may
fail" warning — even though the build succeeds because the symbols are
compiled into the core. Now logs an accurate "provided by arduino-esp32
core — already compiled in, not merging". Same treatment for core
headers that aren't standalone lib dirs (Udp.h, IPAddress.h,
WiFiUdp.h, …) via a new _CORE_ESP32_HEADERS allowlist.
SimulatorCanvas: the WiFi badge's "open IoT gateway" click now consults
an optional window.__velxio_iot_gateway_open_gate__ hook before opening
the gateway tab. A private overlay can install it to gate the gateway
behind a paid plan and show an in-place upgrade modal instead of dumping
a 402 page in a new tab. OSS builds have no hook → opens normally. The
check is synchronous so it doesn't trip popup blockers.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds a generic gating hook so a private overlay can restrict the IoT
gateway proxy to paid plans without the OSS image carrying any plan
logic. register_iot_gateway_gate() installs an async callback that
returns None to allow or a detail dict to block; the OSS default (no
overlay) allows everyone, and a failing gate fails OPEN so the gateway
can never be taken down by a buggy overlay.
gateway_proxy() calls the gate first. When blocked it content-
negotiates the 402: browsers (Accept: text/html — the frontend opens
the gateway via window.open) get a small styled upgrade page with a
link to /pricing; programmatic fetch/XHR callers get the JSON detail.
No behaviour change for the open-source image — the gate is a no-op
there.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Brings hardware flashing into Velxio Desktop. Per-board "Flash to
real board" entry in the canvas context menu opens a modal that
enumerates USB serial ports, lets the user pick one, then
streams arduino-cli upload output live until the board is flashed.
Backend (Phase D1) — backend/app/api/routes/flash.py (new):
POST /api/flash/upload (multipart: board_id, port, fqbn,
program_format, program)
→ SSE stream of {phase, line?, progress?} events
→ final {phase:'done', success, elapsed_ms, error?}
- Wraps `arduino-cli upload -p <port> -i <file> --fqbn <fqbn> -v`
so AVR (avrdude), ESP32 (esptool), RP2040 (picotool), SAMD
(bossac) all share one code path — arduino-cli internally
dispatches by FQBN.
- Per-port asyncio.Lock prevents two simultaneous flashes from
fighting over the same /dev/ttyACM0.
- Allow-list of FQBN prefixes (arduino:avr, ATTinyCore:avr,
rp2040:rp2040, esp32:esp32, arduino:samd) so a typo can't
cause a confusing arduino-cli error.
- Format allow-list (hex / bin / uf2 / elf) drives the temp
file extension - arduino-cli uses the extension to route to
the right uploader.
- 8MB hard cap on the uploaded program (real sketches are
well under that; protects against a runaway frontend).
- X-Accel-Buffering: no header so nginx doesn't hold the SSE
chunks until the flash completes.
Frontend (Phase D3):
- frontend/src/services/flashService.ts (new):
async generator streamFlash() yields parsed SSE events.
Handles the base64-vs-text gotcha (compile returns hex_content
as text but binary_content as base64; for binary formats we
atob() into a Uint8Array before posting so the form upload
sends actual bytes, not the base64 ASCII).
- frontend/src/components/simulator/FlashModal.tsx (new):
Three-state UI: picking (port dropdown), flashing (progress
bar + live log), success/error (verdict + retry).
Empty-ports state shows a Linux dialout-group hint.
- SimulatorCanvas.tsx: board context menu gains "Flash to real
board" entry, gated on isTauri() + presence of compiledProgram.
Hidden in web (WebSerial is a separate sprint).
- tauriBridge.ts: SerialPortInfo type + listSerialPorts() helper
that invokes the Rust shell command added in Phase D2.
The sidecar already has arduino-cli on PATH (per
`pro/desktop/sidecar/main.py::_expose_bundled_arduino_cli`), so
no installer changes are needed — flash works the moment the
0.4.x desktop bundle ships with these commits.
Plan + remaining phase tracked in project/hardware-flashing/.
D2 (Rust serial enum) committed separately as a Tauri-shell-only
concern; D4 (manual smoke matrix with real boards) requires
physical hardware so it stays a follow-up.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The desktop Tauri wrapper drops the downloaded libqemu-xtensa.<ext>
inside the directory it exports as VELXIO_QEMU_PATH. The sidecar
previously only checked the dedicated QEMU_ESP32_LIB env var (full
path) plus a fixed file beside the module, so the in-app installer's
output was invisible — adding an ESP32 to the canvas right after
install reported "unavailable" until the sidecar was restarted.
Two changes:
1. New _resolve_lib() helper checks QEMU_ESP32_LIB first, then
VELXIO_QEMU_PATH/<lib_name>, then the legacy module-adjacent path.
Both Xtensa and RISC-V flow through the same resolver.
2. lib_xtensa_path() / lib_riscv_path() functions replace the module
constants for all internal call sites (is_available,
is_riscv_available, start_instance). Resolution happens per call,
so a post-boot install is picked up without restarting the
sidecar. The LIB_PATH / LIB_RISCV_PATH constants stay for any
external readers but reflect import-time state only.
Pairs with the Tauri-side change that writes libqemu-xtensa.<ext>
directly to VELXIO_QEMU_PATH (no archive extraction).
User report after the v0.4.0 desktop agent landed:
> agente devuelve "LLM call failed: Failed to fetch"
"Failed to fetch" is a network-layer error, not 401. Root cause: the
OSS CORS allow_origins list only included http://localhost:517[3-5]
(vite dev) and settings.FRONTEND_URL. The desktop bundle runs from
either tauri://localhost (macOS/Linux) or http://tauri.localhost
(Windows) - both cross-origin to velxio.dev - so the browser
blocked the agent's POST /api/pro/agent/llm preflight before the
backend ever saw it.
Added all three Tauri scheme variants to the allow list. After this
lands + a backend restart the desktop agent's fetch reaches the
real /api/pro/agent/llm and the dual-auth dep from v0.4.0 Phase 1
gets to do its job (Bearer license-key → resolved User → quota
check → upstream LLM proxy).
Origins added:
tauri://localhost # macOS / Linux (Tauri 2.x default)
http://tauri.localhost # Windows (Tauri 2.x default)
https://tauri.localhost # older Tauri 2.x releases
allow_credentials stays True - the existing cookies-from-web flow
still works, the Tauri origins just don't have any cookies to send.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A 1024-byte oled.show() writevto generates ~1025 calls into
_on_i2c_event, one per byte. The previous code called _log (stderr
write+flush) AND _emit (stdout JSON write+flush) for every event,
saturating the worker subprocess's stdout pipe. The QEMU thread blocks
on the synchronous write, the firmware's ESP-IDF i2c_master ISR
re-enters before the previous one finished, and the Interrupt watchdog
trips on CPU1 with a "Guru Meditation Error: Interrupt wdt timeout"
panic on the second consecutive oled.show() call.
I2CWriteSink already buffers writes internally and emits a single
i2c_transaction event on FINISH, so the per-byte log+emit was pure
overhead with no observability value for display drivers (SSD1306,
PCF8574). Keep them for everything else.
Verified end-to-end via chrome devtools MCP:
- Test minimal (init + 1 explicit show): markers OK, no panic
- Loop test (10x oled.show() + sleep(0.5)): DONE_LOOP, no panic
ESP-IDF runs component_get_requirements.cmake in script mode (not full
project mode) to scan component dependencies before the build proper.
CMake rejects CONFIGURE_DEPENDS in script mode with:
CONFIGURE_DEPENDS is invalid for script and find package modes.
The persistent build dir is already wiped + restored from the template
on every compile (espidf_compiler.py:220-222 + CMakeLists.txt patches),
so the glob is re-evaluated naturally on the next cmake configure — we
don't need CONFIGURE_DEPENDS to catch new files.
Pairs with 27c9a28 (glob fix) + 21791a8 (template main.cpp decouple).
Now that main/CMakeLists.txt globs every *.cpp/*.c into SRCS (commit
27c9a28 — needed so multi-file Arduino projects link), sketch.ino.cpp
is compiled as its own translation unit. The template main.cpp's
`#include "sketch.ino.cpp"` is left over from when SRCS only named
main.cpp — back then we had to pull the user code into main.cpp's TU
to get it compiled at all.
With the glob in place, keeping the include re-defines setup() and
loop() in main.cpp's TU AND in sketch.ino.cpp.obj, so the linker dies
with "multiple definition of `setup'". Replace the include with a
forward declaration and let the linker resolve setup()/loop() from
sketch.ino.cpp.obj. Matches arduino-cli's per-file compile + auto-link
model, and frees pure-C++ multi-file sketches to put helper TUs in
their own .cpp files without textual-include hacks.
Repro: robot-desktop-eyes after the glob fix (#13 rebuild) — compile
got past resolve+compile and into link, where multi-def of setup()
would have appeared on the next attempt. This commit closes the loop.
Multi-file Arduino sketches (robot-desktop-eyes is the canonical
case) define classes and free functions in their own translation
units — Face::Update(), FaceExpression::GoTo_Surprised(),
AsyncTimer::SetIntervalMillis(...), etc. live in Face.cpp,
FaceExpression.cpp, AsyncTimer.cpp respectively.
The espidf_compiler drops every user-supplied .h/.cpp into
project/main/ but the main/CMakeLists.txt only declared
`SRCS "main.cpp"`, so the helper TUs never got compiled and the
linker died with dozens of "undefined reference to ..." entries.
Switch SRCS to a CONFIGURE_DEPENDS glob over the main/ directory
so every .cpp/.c that lands in main/ becomes part of the IDF main
component automatically. CONFIGURE_DEPENDS makes CMake re-evaluate
the glob on every reconfigure, which matters because the persistent
build dir is reused across compiles and the set of helper files
changes per sketch.
Repro: open robot-desktop-eyes, click Compile. Before this commit:
37 linker errors starting at "undefined reference to `u8g2'" and
"undefined reference to `Face::Update()`". After: the helper TUs
compile and the .elf links.
Two related fixes for the ESP32 Arduino-compat compile path:
(a) backend/app/services/esp-idf-template/main/CMakeLists.txt:
Demote -Werror=comment / =parentheses / =sign-compare / =narrowing
/ =write-strings / =missing-field-initializers / =reorder back to
plain warnings. ESP-IDF's project defaults are stricter than what
Arduino/arduino-cli users expect, so common Arduino idioms (nested
/* */, missing field initializers in struct literals, etc.) were
failing builds that compile fine in the Arduino IDE. -Wall stays
on; we just stop the abort.
(b) examples-robot-desktop.ts (robot-desktop-eyes example):
Replace the nested /* xTaskCreatePinnedToCore( ... /* Task function. */
... */ block with `#if 0 / #endif` so the inner block comments
don't terminate the outer one. Even with -Wno-error=comment the
real-syntax-level issue (the first inner `*/` closes the outer
comment, leaving the rest of the lines as bare code) would still
bite, so this needs an actual code fix.
When a sketch's external library headers were only referenced from
project headers (e.g. esp32-eyes.ino includes Common.h, Common.h
includes <ESP32Servo.h>), the compile failed with
fatal error: ESP32Servo.h: No such file or directory
because _detect_external_includes was only called on main_content
(the processed .ino). Project .h/.cpp files were never scanned, so
ESP32Servo / DHT / Adafruit_Sensor referenced only transitively
through user code never reached _resolve_library_components and
never landed in user_libs_all/.
Fix: collect ext_headers from main_content PLUS every uploaded
.h/.hpp/.ino/.c/.cpp file before resolving libraries. Lib resolver
already walks transitive includes inside the lib bundle once it's
copied; this just makes sure the first-level set covers user
project headers too.
Repro: open https://velxio.dev/example/robot-desktop-eyes, click
Compile. Before this commit: 13 errors starting at ESP32Servo.h.
After: ext_headers includes ESP32Servo.h on the first pass and the
build proceeds.
Two intertwined bugs were leaving every ESP32 ePaper example broken
end-to-end. Only the 5.65" UC8159c panel surfaced the failure
audibly ("Busy Timeout!" repeating in serial), because its inverted
busy polarity caused the firmware to hang inside `_waitBusy()`. The
SSD168x ePaper examples APPEARED to run cleanly but never actually
rendered anything to the panel — the canvas stayed at the idle paper
colour because the same registration path was broken.
Root cause #1 — `setSensors` was a full REPLACE, not a merge.
`Esp32Bridge.setSensors(sensors)` did `this._pendingSensors =
sensors`. At `startBoard()` time the store iterates components,
resolves wires for any entry in `SENSOR_COMPONENT_MAP` (DHT22 /
HC-SR04 / I²C sensors) and calls `setSensors(...)` with that list.
ePaper components live in `PartSimulationRegistry` (not in the
sensor map) and are registered via `sendSensorAttach()` AT
COMPONENT-MOUNT TIME — well before `startBoard()` runs. Full-replace
semantics blew that registration away on every Run click, so the
worker never instantiated an `Ssd168xEpaperSlave` / `Uc8159cEpaperSlave`,
no SPI bytes were decoded, no frames were latched, and BUSY was
never driven.
Fix: upsert by `pin` so pre-existing registrations from
PartSimulationRegistry handlers are preserved alongside the
startBoard-resolved sensors. Confirmed via a WebSocket spy that the
`start_esp32` payload now carries the ePaper sensor entry.
Root cause #2 — BUSY polarity was hard-coded for SSD168x only.
Verified against upstream GxEPD2 source:
* SSD168x family — constructor passes `_busy_level = HIGH`
→ BUSY=HIGH means busy, LOW means ready.
* UC8159c family — constructor passes `_busy_level = LOW`
→ BUSY=LOW means busy, HIGH means ready.
The worker only drove BUSY after a frame flush (and at the wrong
polarity for UC8159c), so the firmware's first `_waitBusy()` inside
`_PowerOn()` / `_InitDisplay()` — which fires BEFORE any frame —
blocked for the full 25 s `_busy_timeout`.
Fix: read `controller_family` from the registration payload, pick the
per-family idle level, and (a) seed the pin to IDLE at registration so
the first `_waitBusy()` sees "ready" immediately, (b) use that
polarity (idle vs. busy) when pulsing on frame flush.
Verified on https://velxio.dev/example/epaper-5in65-7c-esp32-rainbow:
the serial timeline now reads `_InitDisplay reset : 1566` /
`_PowerOn : 148` / `_PowerOff : 183` / `frame done` (all sub-2 ms
busy-waits, no timeouts). Sensor registration confirmed via the
`start_esp32` payload carrying the `epaper-ssd168x` entry.
The user_libs_all bundler in _resolve_library_components does BFS over the
sketch's external includes, copying each matching Arduino library into
one merged IDF component. Anything not in _BUILTIN_HEADERS is treated as
an external library to resolve, and the lookup just scans
/root/Arduino/libraries/ for a directory whose `src/` (or root) holds a
matching header file.
_BUILTIN_HEADERS only listed C headers (stdint.h, stdio.h, …). The C++
wrappers (cstdint, cstdio, cmath, …) and the STL containers (vector,
complex, string, …) were absent. Result: any library transitively
#including <cstdint> or <vector> caused the bundler to "resolve" the
header against /root/Arduino/libraries/ArduinoSTL/ — an AVR-only
uClibc++ port that ships every C++ stdlib header as plain files. Once
ArduinoSTL was matched the bundler dragged in ALL of it, including
complex.cpp:
template class _UCXXEXPORT complex<float>;
which fails on the ESP-IDF Xtensa toolchain because _UCXXEXPORT isn't
defined in that compile context AND the symbol already exists in the
real libstdc++ pulled in by <complex>. Net effect: every ESP32 sketch
whose deps transitively include a C++ stdlib header (e.g. ESP32Servo
includes <cstdint>) blew up with 66+ errors before the servo example
even reached the link step.
Fix: extend _BUILTIN_HEADERS to cover the full set of C++ stdlib
wrappers and STL headers so the bundler never treats them as installable
libraries. The Xtensa GCC + libstdc++ shipped by ESP-IDF provides them
natively; ArduinoSTL never has any business being part of an ESP32 build.
Verified end-to-end on /example/esp32-servo: compile now succeeds, sketch
boots, moving the potentiometer drives the wokwi-servo angle (Pot=2801 →
Angle=123 deg, servo arm rotates).
arduino-esp32 3.x ledcWrite takes a PIN and looks up the attached channel
internally. arduino-esp32 2.x (the toolchain version we pin) takes a
CHANNEL. The velxio_compat.h shim already aliased the 3.x-only
ledcAttach onto ledcSetup+ledcAttachPin so 3.x sketches would compile,
but ledcWrite still mapped 1:1 — so a call like
#define R_PIN 16
ledcAttach(R_PIN, 5000, 8); // shim → channel 0 attached to pin 16
ledcWrite(R_PIN, 128); // ★ writes to "channel 16" (invalid)
silently wrote to LEDC channel 16, which doesn't exist (valid range
0-15). The hardware duty register never changed, qemu-lcgamboa never
emitted a `ledc_duty` event, and the RGB LED stayed dark even though
the firmware ran cleanly and the wires looked right. Verified end-to-end
with examples/esp32-pwm-led-rgb: gpio_change events fired at boot, no
ledc_duty events fired, ledRed/ledGreen/ledBlue all stayed at 0.
Fix: maintain a 40-entry pin→channel table populated by both ledcAttach
variants. Replace ledcWrite with a macro that calls a helper checking
the table first; if the value isn't a known pin we pass it through as a
channel, preserving 2.x channel-style call sites.
Macro/function name collision is sidestepped with the standard
parenthesizing trick — `(ledcWrite)(channel, duty)` doesn't expand the
function-like macro because the token isn't followed by `(`.
Verified live on velxio.dev/example/esp32-pwm-led-rgb after hot-copying
the new header into the velxio-app container: ledRed/ledGreen/ledBlue
now cycle through the full HSV wheel as expected (samples: (255,41,0),
(41,255,0), (0,41,255), (232,255,0), …).
Single-file sketch only — the table is `static` (internal linkage) and
ledcAttach + ledcWrite live in the header. Multi-file sketches that
attach in file A and write in file B would each see their own table.
Acceptable for now since arduino-esp32 sketches are nearly always
single-file; revisit when we bump the toolchain to 3.x and can drop the
shim entirely.
ATTinyCore >=1.5.0 declares ATTinyCore:micronucleus@2.5-azd1b as a tool
dependency, hosted at https://azduino.com/bin/micronucleus/. That host
has been unreachable (connection refused) for extended periods, causing
every ATtiny85 compile to fail at the core-install step with:
Download failed: performing HEAD request: ... dial tcp ...: connection refused
Failed to install required core: ATTinyCore:avr
micronucleus is only used for USB upload — never for compilation — but
arduino-cli refuses to install a core whose tool deps cannot fetch.
Pin to 1.4.1, the last release whose micronucleus binary is hosted on
github.com (digistump release, reachable). The FQBN clock options we
ship (clock=16pll on attinyx5, etc.) are unchanged across 1.4.x.
- backend/app/services/arduino_cli.py: new CORE_INSTALL_VERSIONS map
consulted by ensure_core_for_board so the runtime auto-install
passes "ATTinyCore:avr@1.4.1" instead of unversioned latest.
- backend/Dockerfile and docker/entrypoint.sh: same pin so a fresh
image bakes 1.4.1 in and never hits the runtime fallback path.
Existing regression tests in test/backend/unit/test_arduino_cli_attinycore.py
still pass (they assert presence, not version).
Two CI failures landed after PR #196 (esp32-gpio-matrix-cb-callback)
merged. Both are independent and fixed here together.
1) **Backend E2E: ESP32 hangs at bootloader handoff.**
PR #196 added picsimlab_gpio_matrix_cb which fires on QEMU's
iothread. The handler did `_emit({...})` for every routing
change — and the ESP-IDF bootloader writes to gpio_out_sel
*hundreds* of times during early boot (each peripheral init
configures its matrix slot). Each emit acquires _stdout_lock
and writes to the worker→manager pipe. If the manager drains
even briefly slow, the pipe fills, write blocks, and the
iothread stalls — symptom: ESP32 reports `entry 0x400805e4`
then no Arduino setup() output for 75 s.
Fix: the iothread callback now ONLY mutates the SignalRouter
snapshot. It never emits. The 10 Hz poll thread
(_refresh_signal_routing) stays as the sole emitter, so the
wire-format event stream is unchanged. Benefit of having the
callback over poll-only is reduced worst-case routing-emit
latency (next poll tick vs up to 100 ms) and a warmer
snapshot dict for cheaper poll diffs.
2) **Frontend Tests: Node OOM at end of suite.**
117 test files run in one forks-pool worker. Several lazy-load
the ngspice emscripten module (~30 MB), the MixedModeScheduler
singleton, and other heavy modules whose dispose hooks aren't
reached because singletons leak across files. Cumulative heap
pressure exceeds Node's 4 GB default; the worker hits "Ineffective
mark-compacts near heap limit" AFTER all 1881 tests pass and
the OOM kill is reported by vitest as "Worker exited unexpectedly
/ Timeout terminating forks worker". This is not a real test
failure — every individual test passes.
Quick fix: pass NODE_OPTIONS=--max-old-space-size=8192 to the
`npm test` step. Long-term, the singletons should add dispose
hooks that test fixtures call in afterAll(), or the suite
should shard into multiple `vitest run --shard` invocations.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires the new synchronous GPIO Matrix callback exposed by
libqemu-{xtensa,riscv32} 1.1.0 (lcgamboa/qemu commit e178ff5).
Whenever the firmware writes GPIO_FUNCx_OUT_SEL_CFG_REG, the C
plugin now fires picsimlab_gpio_matrix_cb(gpio, signal_id) inline.
The handler:
- Treats signal_id == 0x100 or 0 as "matrix routing cleared" and
emits gpio_routing_clear.
- For LEDC HS/LS range signals (the only ones the frontend
SignalRouter currently consumes), updates the mirror and emits
gpio_routing.
- Drops other signals — the mirror does not need to track them
yet, and emitting them would only fatten WS frames.
Backwards compat:
- Older libqemu (<1.1.0) doesn't expose the new field; the
picsimlab_gpio_matrix_cb placeholder runs (no-op) and the
100 ms _refresh_signal_routing() poll thread continues to feed
the mirror. WS event shape is identical either way.
Burn-in: keeping the poll thread active in parallel with the
callback for now. Once telemetry confirms parity (per phase 4 doc
in velxio-prod/project/esp32-gpio-matrix-cb/), the poll thread
gets retired in a follow-up commit.
Adds a third format to /api/compile-rom: `c` (C source compiled by SDCC
to Z80 bytes). Same chip-program flow as 8080/Z80 asm — write C in a
project file, click Compile, click Run.
Backend:
- backend/app/services/c_compile.py — async SDCC wrapper. Locates the
sdcc binary on PATH (or via SDCC env var, or common Windows install
paths) and shells out with target=mz80 + --code-loc 0x100 --data-loc
0x8000. Parses the resulting Intel HEX into raw ROM bytes. Pure 8080
is rejected with a clear error (SDCC has no 8080 backend; Z80 ROMs
also run on the i8080-cpu chip if you avoid Z80-only ops).
- rom_compile.py: compile_rom is now async; the new c branch delegates
to c_compile. compile_rom_endpoint awaits it.
Frontend:
- romCompileService: RomFormat gains 'c'; formatForFile maps .c/.cpp to
'c'. isChipProgramFile intentionally still excludes .c — disambiguation
happens at the EditorToolbar level.
- EditorToolbar: the chip-program path also fires when a custom-chip
has programFile === activeFile.name (regardless of extension). That
lets .c files route to /api/compile-rom (SDCC) when bound to a CPU
chip, while .c files NOT bound to any chip continue to route to
arduino-cli as before.
Docker:
- Dockerfile.standalone adds `sdcc` to the apt-get install list, so the
prod image ships with SDCC out of the box.
Example:
- /examples/z80-led-chaser-c — z80-cpu chip + chaser.c (a Larson
scanner written in C with __at() MMIO definitions). Compiles cleanly
with SDCC's --code-loc 0x100 default crt0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the Zilog Z80 to the programmable-retro-CPU lineup. Same compile-rom
flow that landed for the 8080 in PR #189: write Z80 asm in a project
file, click Compile (backend assembles via in-tree two-pass asm-z80),
click Run, the chip emulator boots from the resulting ROM bytes.
Backend:
- backend/app/services/asmz80.py — two-pass Z80 assembler covering the
practical demo subset: LD r,n / r,r' / rp,nn / (nn),A / A,(nn) +
ALU r/n + INC/DEC + JP/JR/DJNZ/CALL/RET + PUSH/POP + IN/OUT +
EX/EXX + LDIR/LDDR/IM/NEG + RLCA/RRCA/RLA/RRA + the simple
ED-prefix variants. Not yet: CB-prefix bit ops, DD/FD index ops.
- rom_compile.py routes target=z80 through the new assembler.
Chip:
- frontend/src/components/customChips/examples/intel/z80-cpu.{c,chip.json}
Generated by scripts/make-z80-cpu.py from the existing z80.c emulator
(same clean-room implementation that passes ZEXDOC end-to-end). The
external pin/bus protocol is replaced with internal RAM + ROM + MMIO
for LED/BTN/UART. 35 KB WASM.
Example:
- /examples/z80-larson-scanner — Knight-Rider-style walking LED.
Demonstrates JR/DJNZ/RLCA which the 8080 can't run.
Plus a small Z80 smoke-test asm under scripts/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a new way to use the retro CPU chips: write your program in a
project file (.s / .asm / .hex / .bin), click Compile, click Run, and
the same chip emulates whatever you wrote. Same chip + different ROMs =
mini PC, calculator, LED demo, Kill-the-Bit game, etc.
SDK:
- velxio-chip.h gets two new host imports:
uint32_t vx_rom_size(void);
void vx_rom_read(uint32_t off, uint8_t* dst, uint32_t len);
CPU-emulator chips call these in chip_setup to pull their program out
of the host's romBytes property.
Frontend runtime:
- ChipRuntime accepts opts.romBytes (Uint8Array) and exposes the new
imports, copying bytes into chip memory on vx_rom_read.
- CustomChipPart pulls component.properties.romBytes (base64) and passes
it through.
- Component registry declares three new custom-chip properties:
romBytes (base64), programFile (matching project filename), and
programTarget (cpu name).
New programmable bundled chip:
- frontend/src/components/customChips/examples/intel/i8080-cpu.{c,chip.json}
Same clean-room 8080 emulator as i8080-repl/i8080-counter, but ROM is
loaded externally via vx_rom_*. Has 8 LEDs, 8 buttons, UART, 16 KB RAM,
32 KB of external ROM.
Backend:
- New /api/compile-rom endpoint and rom_compile service that turns
chip-program source into ROM bytes. 8080 ASM is assembled by the
in-tree two-pass assembler (moved to backend/app/services/asm8080.py).
Intel HEX records are parsed; raw .bin is passed through. Future targets
(z80, 8086, 4004) are scaffolded but not wired yet.
EditorToolbar:
- Compile button detects when the active file is .s/.asm/.hex/.bin and
routes to compile-rom instead of arduino-cli. The compiled bytes are
injected into every custom-chip on the canvas whose programFile property
matches the active filename (or is empty).
Example:
- /examples/i8080-killbits loads Dean McDaniel's 1975 Kill-the-Bit on
the programmable i8080-cpu chip. killbits.s is shipped as a project
file alongside sketch.ino; the user clicks Compile then Run and the
LED walks across 8 outputs, buttons kill it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The SignalRouter path has been in prod through Phase 2.5 / Phase 3.3
deploys without regressions, so the temporary fallback shipped in
commit 77bf897 can come out. Closes#101.
Backend (esp32_worker.py + esp32_lib_manager.py):
- Stop emitting `ledc_update` from the 0x5000 LEDC callback and from
the polling thread. Only `ledc_duty` (channel + duty_pct) and the
GPIO matrix routing events ship now.
- Drop the channel→gpio reverse-lookup that fed the legacy event.
Frontend:
- Delete `PinManager.broadcastPwm` and `PinManager.pwmListenerPinCount`.
- Delete `makeLedcUpdateHandler` + its `channelGpioMemo`.
- Delete `Esp32Bridge.onLedcUpdate` field + the `case 'ledc_update':`
message handler + the `LedcUpdate` type.
- Strip `this.onLedcUpdate = null` from 14 test mocks.
- Rewrite the `does not call broadcastPwm` guard in
esp32-multi-servo-gpio-matrix.test.ts to assert the method itself
no longer exists on PinManager (stronger regression guard than the
spy version, and doesn't need vi).
- Remove the `PinManager.broadcastPwm fallback` describe block from
esp32-servo-pot.test.ts — every test in it exercised the deleted
fallback path.
Docs (ESP32_EMULATION.md):
- Replace `ledc_update` rows in the events / implementation tables
with the SignalRouter trio (`ledc_duty`, `gpio_routing`,
`gpio_routing_clear`).
- Update the visual flow diagram + the "why this matters" paragraph
to past-tense the broadcastPwm bug.
Tests: 1886 frontend tests pass (the previously-failing
board-kinds-coverage test that needed the new Pi Zero/1/2 kinds is
also green). Backend unit suite: 279 pass, the 11 espidf_real_paths
prereq failures are environment-dependent (need arduino-cli libs in
the local shell) and unrelated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
End-to-end pipeline fixes uncovered while auditing the /examples gallery.
Each bug shipped past green unit + snapshot tests because none of those run
firmware + render LEDs. Added scripts/visual-led-test.mjs as a CDP-driven
visual harness that loads each example, runs the simulator, samples
`wokwi-led.brightness`, and asserts toggle / gradient / initial-off
invariants — exits non-zero on any regression.
Frontend simulator
- PinManager.updatePort: new optional ddrMask param. A pin is added to
`outputPins` only if the DDR bit is set, so the PORTx write that
enables INPUT_PULLUP (DDR=0, PORT=1) no longer falsely marks the pin
as MCU output. AVRSimulator now reads DDRB/C/D (0x24/0x27/0x2A on
Uno/Nano, 0x37 on ATtiny85, per-port table on Mega) and forwards it.
- AVRSimulator: pass DDR mask alongside every port-listener fire.
- BasicParts pushbutton{,-6mm}: seed pin HIGH in attachEvents so
`digitalRead()` returns HIGH while idle. avr8js doesn't auto-simulate
INPUT_PULLUP — without this the firmware reads LOW from boot and
thinks the button is permanently pressed (the "LED is always on,
pressing does nothing" UX bug).
- connectMcuEdgesToService: suppress synthetic digital edges on pins
with active PWM, AND subscribe to onPwmChange to re-tick the netlist
on duty changes. Fade-LED now produces a true gradient (6 distinct
brightness levels across a fade cycle) instead of a binary 0/full
toggle.
- CircuitSimulationService.handleMcuEdge: replace single-slot
pendingMcuEdge with a per-pin Map. Multiple pins toggling during the
same in-flight tick used to overwrite each other; now every pin's
most-recent edge replays after the tick. Fixes Traffic-Light RED→
YELLOW→GREEN sequencing.
- NetlistBuilder: new sanitizeSpiceId() helper replaces hyphens with
underscores in V-source names. ngspice's interactive `alter` command
treats `-` as an operator and silently no-ops on hyphenated source
names, so mid-simulation MCU pin transitions stopped propagating
after the first solve. MixedModeScheduler.onMcuPinChange and
CircuitSimulationService self-heal use the same sanitizer so names
stay consistent across emit/alter/lookup. Also added a regex-based
fallback in step 2 so any board pin matching `GND.\d+` canonicalises
to net "0" — ESP32-C3 dev kits expose up to 10 GND pins and the
per-board `groundPinNames` list missed several, leaving wires
floating instead of grounded.
- collectPinStates: emit V-sources only for pins in `outputPins`, not
every wired board pin. Leaves INPUT pins (analog sensors on A0,
pull-down dividers, etc.) free for the SPICE solver instead of being
shorted to 0 V by an ideal MCU V-source.
- start.ts: extended __spiceDebug to also expose outputPinsByBoard +
nodeVoltages + pinNetMapEntries for the visual harness.
- ESP32 / RP2040 / RISC-V / C3 simulators: pass `'mcu'` source flag to
triggerPinChange / setPinState so the new outputPins tracking fires
on those boards too (was AVR-only before).
- useSimulatorStore: stopBoard/resetBoard call pm.resetPinStates() so
outputPins clears between runs; Esp32Bridge.onPinChange passes the
`'mcu'` flag in all three places it's wired.
- types/board.ts: ATtiny85 FQBN `clock=internal16mhz` →
`clock=16pll` (ATTinyCore 1.5.2 renamed the option).
Backend
- esp-idf-template/main/CMakeLists.txt: skip the
`-DLED_BUILTIN=2` fallback for esp32c3 and esp32s3 targets. Both
variants already define LED_BUILTIN in pins_arduino.h via a
self-define macro (`#define LED_BUILTIN LED_BUILTIN` + `static const
uint8_t LED_BUILTIN = ...;`). Pre-defining the symbol from the
command line expanded the static-const declaration to
`static const uint8_t 2 = ...;` — a syntax error that broke every
ESP32-C3 / S3 build (`expected unqualified-id before numeric
constant`).
Examples
- examples.ts: bulk-fix 72 wire endpoints that referenced
`componentId: 'nano-rp2040'` / `'esp32-c3'` etc. (boards that don't
exist on the canvas). Replaced with `'arduino-uno'` (the canvas
board-id convention) and converted `D<n>` pin names to `GP<n>` for
Pico-style boards. Affects pico-blink, pico-i2c-scanner,
pico-i2c-rtc-read, pico-spi-loopback, c3-blink and others.
Tests
- scripts/visual-led-test.mjs: CDP-driven harness. Default suite covers
Blink (single-pin), Button (idle-OFF invariant — catches the
INPUT_PULLUP regression), Traffic-Light (multi-pin sequencing),
Fade-LED (PWM gradient — ≥3 distinct levels), RGB-LED (≥3 PWM pins
driven). Run via `npm --prefix frontend run test:visual` against a
Chrome on `:9222` + vite on `:5174` + backend on `:8001`.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Backend:
- api/routes/compile.py accepts board-specific compile options
and dedups in-flight identical requests
- services/espidf_compiler.py expanded ESP-IDF wrapper with the new
options surface (sdkconfig.defaults.in
template added)
- services/arduino_cli.py honour the new options envelope
- services/esp32_lib_bridge.py thread board options through to QEMU
Tests:
- tests/test_compile_request_dedup.py end-to-end dedup behaviour
- tests/test_espidf_options.py covers the new options parsing
Frontend:
- services/compilation.ts client-side mirror — sends the new
options field on every compile request
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the deferred Phase 3.3. Root-causes the Pi 2 "Attempted to
kill init" panic as `mount /dev/vda` failing with EINVAL — Debian
armmp does not have ext4 builtin (only fuseblk in /proc/filesystems).
- qemu_manager: PI_CONFIGS gains raspberry-pi-zero / -1 / -2 entries.
All three use the armmp armhf kernel + Cortex-A7 CPU + the mmio
virtio transport (arm-32 virt PCI fails -75 due to missing reg DT
property). Pi Zero / Pi 1 get the small 1-core / 512 MB profile;
Pi 2 gets 4-core / 1 GB. QEMU command builder branches on cfg.bus
for virtio-blk-pci vs virtio-blk-device (and serial likewise).
- manifest.json: new `raspberry-pi-armhf` image_set wiring three
assets (kernel + initramfs + zstd rootfs).
- Frontend BoardKind gains the three new kinds + an isPiBoardKind()
helper. Replaces the eight scattered `=== 'raspberry-pi-3' ||
=== 'raspberry-pi-4' || === 'raspberry-pi-5'` branches in
useSimulatorStore, Interconnect, loadExample, boardProtocols.
ComponentRegistry gets three new picker entries.
- board-kinds-coverage test: ACCEPTED_UNCOVERED gains the new kinds
(backend boards have no canvas examples).
The matching armhf build-pi-kernel.sh / build-pi-rootfs.sh changes
live in velxio-prod's scripts/ (private overlay) — the upstream
kernel build script only knows about arm64; armhf is built in the
private repo because the assets ship through the license endpoint.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the public extension points the velxio-prod overlay uses to bind
real canvas-side I2C/SPI/UART models (BME280, future MCP23017, etc.)
to a running Pi guest's protocol shims:
- qemu_manager: set_pi_slave_handler(fn) / get_pi_slave_handler() for
pi_attach_slave + pi_detach_slave WebSocket messages. OSS image
leaves the hook unset so the messages are silently dropped.
- simulation route: parses the two new WS message types and forwards
them to the registered handler when present.
- RaspberryPi3Bridge: attachSlave(spec) / detachSlave(spec) frontend
side of the protocol.
- piSlaveScanner: at simulation start walks components + wires,
identifies I2C/SPI/UART peers wired to Pi protocol pins (40-pin
header physical-pin numbering), and emits one attach per
bus/address pair (deduped across SDA+SCL wires).
- RaspberryPiWorkspace: invokes the scanner once the bridge is open,
with retries to ride out the WS-still-connecting race.
- integration test: pi3_bme280_attach.py boots the Pi, pre-attaches a
BME280 via the slave handler, runs a host-side proto loop, runs
guest python smbus2.read_byte_data(0x76, 0xD0) and asserts the
console reads back CHIP=0x60.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Backend: extract per-board config into a PI_CONFIGS dict keyed by
board_type. Pi 3/4/5 share the same arm64 image set (kernel +
initramfs + rootfs) and differ only in QEMU -cpu and -m:
raspberry-pi-3 → cortex-a53 + 1G (BCM2837, ARMv8 64-bit)
raspberry-pi-4 → cortex-a72 + 2G (BCM2711, ARMv8 64-bit)
raspberry-pi-5 → cortex-a76 + 2G (BCM2712, ARMv8 64-bit)
PiInstance now carries board_type so the per-board lookup happens
once at start_instance time. Unknown board_type falls back to
DEFAULT_PI_BOARD ('raspberry-pi-3') instead of erroring out (for
back-compat with older clients).
Pre-warm hook walks every unique image_set in PI_CONFIGS so the
provider only downloads each set once even when several Pi models
are registered.
Frontend:
- BoardKind union gains 'raspberry-pi-4' and 'raspberry-pi-5'.
- BOARD_KIND_LABELS + BOARD_KIND_FQBN entries for both new boards
(FQBN null since they use the Pi VFS + Python toolchain like Pi 3).
- ComponentRegistry inserts two new component metadata entries
cloning the Pi 3 board art with different thumbnail colours.
Tag name reused so the same velxio-raspberry-pi-3 web element
draws the board on the canvas — the 40-pin GPIO layout is
identical across Pi 3/4/5.
- boardProtocols.ts: Pi 3/4/5 share the BCM physical→GPIO table
(PI3_BCM) since the 40-pin header layout is identical.
- loadExample.ts: where 'raspberry-pi-3' is special-cased (VFS
ingest, .cpp vs .ino filename), now matches Pi 3/4/5 alike.
- Interconnect.isPi3Bridge() recognises all three Pi family members
so Arduino↔Pi serial routing keeps working.
- RaspberryPi3Bridge constructor gained a boardKind parameter
defaulting to 'raspberry-pi-3'. The WebSocket 'start_pi' message
now ships the actual board kind so the backend knows which
PI_CONFIGS entry to use.
- useSimulatorStore.addBoard wires bridge construction for all
three Pi family members.
Pi Zero/Pi 1/Pi 2 (armhf) come in Phase 3.3 — separate kernel
package + armhf rootfs build, no change here.
Smoke-tested inside the prod container:
Pi 4 (cortex-a72) → reached agetty login on hvc0
Pi 5 (cortex-a76) → reached agetty login on hvc0
Both show 'aarch64' in uname -m.
The Phase 2 E2E test was sending the Python GPIO command via
'python3 -c "..."' but bash quote-nesting silently corrupted the
script — the python process started, printed nothing, exited 0, and
the test asserted 'GPIO_SETUP 17 out' was missing in proto bytes
(it never got sent because the python script never ran).
Switch the test to base64-encode the script + pipe through base64 -d
into a file, then execute. Verified end-to-end now:
[test] proto received 36 bytes:
GPIO_SETUP 17 out pud_off
GPIO 17 1
[test] ✓ shim → proto pipeline works
Also bump the rootfs manifest entry to the final Phase 2 build
(d6d4a274 raw / debd1c33 zst, version 2026.05+phase2-shims-final).
Earlier auto-discovery in _transport.py was hanging at import time
on some glob/sysfs interaction. Now hardcoded /dev/vport1p1 which
is the empirical path under -M virt + virtio-blk-pci on slot 0.
QEMU 10's virtserialport on a socket chardev (server=on,wait=off)
silently drops guest→host bytes. Reproduced cleanly: writes from
inside the guest to /dev/vport<N>p<M> succeed (no errno) but the
connected client socket receives 0 bytes. Same bug whether the
client is a single recv loop, multiple threads, TCP or UNIX socket,
or whether QEMU runs as server vs client. virtconsole on the same
socket works fine — only virtserialport is broken.
Workaround: use `pipe` chardev (a pair of named FIFOs created
beforehand by qemu_manager). guest→host through .out flows reliably
in QEMU 10 — verified with manual test: 'echo PIPE_TEST > /dev/vport1p1'
in the guest produces 'PIPE_TEST\n' immediately on the host side.
Changes:
- qemu_manager._boot: allocate a temp basename, mkfifo .in + .out,
pass to QEMU as 'pipe,path=<base>'.
- qemu_manager._connect_gpio: open both FIFOs O_RDWR | O_NONBLOCK on
host side (O_RDWR keeps the FIFOs open even when guest hasn't
opened its side yet), wire .out into asyncio via loop.add_reader.
- qemu_manager._reply_gpio / _send_gpio: write to .in fd via os.write.
- qemu_manager._handle_gpio_line: extended Phase 1 GPIO-only parser
into a full Phase 2 mux: GPIO/GPIO_SETUP/GPIO_IN/PWM_*/I2C/SPI/UART
with appropriate replies.
- qemu_manager._shutdown: close FDs + unlink the FIFOs.
- manifest.json: bump raspberry-pi-3-virt rootfs to 2026.05+phase2-shims
(the new rootfs ships the velxio shim Python modules under
/usr/lib/velxio-shims/).
raspi3b pl011 RX is broken in QEMU 10 + kernel 6.12 — see
project/pi-emulation/decisions.md for the full debugging trail.
This commit lands Phase 1 of the rebuild: switch the QEMU machine
to virt + cortex-a53, boot the velxio kernel/initramfs/rootfs over
virtio-blk-pci, and expose the user shell on /dev/hvc0 via
virtio-serial-pci + virtconsole.
End-to-end smoke verified: boot → agetty autologin → bash prompt →
echo round-trip returns the typed token. Tested inside the prod
container with QEMU 10.0.8 and our cloud-derived kernel 6.12.88.
What changed:
backend/app/services/qemu_manager.py
PI3_IMAGE_SET -> raspberry-pi-3-virt
PI3_KERNEL_NAME / PI3_INITRAMFS_NAME / PI3_ROOTFS_NAME new
QEMU cmd rewritten end-to-end:
-M virt -cpu cortex-a53 -smp 4 -m 1G
-kernel <velxio-kernel-arm64> -initrd <velxio-initramfs-arm64.cpio.gz>
-drive ... -device virtio-blk-pci (NOT virtio-blk-device — mmio
variant left /dev/vda unregistered)
-nic none -display none -monitor none -serial none
-chardev socket... -device virtio-serial-pci -device virtconsole
(user console -> /dev/hvc0)
-chardev socket... -device virtserialport,name=velxio-protocol
(Phase 2 channel -> /dev/vport0p2)
No -dtb (virt generates its own), no -append init=... (kernel runs
our initramfs which then switch_root to rootfs and exec's its
/sbin/init — Alpine OpenRC).
backend/app/services/boot_images/manifest.json
New image set raspberry-pi-3-virt with three assets uploaded via
the existing license-endpoint pipeline. Old raspberry-pi-3 entry
flagged deprecated:true and kept for one release for rollback.
test/pi3_console_boot/test_pi3_console_boot.py
Updated QEMU argv to match qemu_manager exactly. Markers now look
for the Velxio Pi Simulator MOTD + 'login on hvc0' (autologin
proof). Round-trip echo still required to pass.
PiTerminal didn't call term.focus() on mount, so xterm.js stayed
passive — onData only fires when the DOM element has focus. Users
saw the boot prompt but their keystrokes went to whatever element
held focus when they clicked Run (canvas, code editor), never
reaching the bridge. Calling focus() right after fit() makes the
prompt receive input the moment it's visible.
The qemu_manager change adds INFO-level logging when serial_input
WebSocket messages reach send_serial_bytes — useful diagnostic for
future Pi3 input problems (proves whether bytes reached the backend
before we look at TTY / kernel / PL011 wiring).
User report: on the solar-tracker project (5218f9e3) only one servo
moved and the log showed `ch=0 duty=X% gpio=12` (wrong — servoPan was
attached to GPIO 13) and `ch=1 ... gpio=-1` (servoTilt's channel
never resolved).
Root cause traced through the GPIO Matrix dump: the firmware does
exactly what the Arduino-ESP32 Servo library says — `ledcAttachPin(
13, 0)` writes signal 71 (LEDC_HS_SIG_OUT0) into `gpio_out_sel[13]`,
and `ledcAttachPin(12, 1)` writes signal 72 (LEDC_HS_SIG_OUT1) into
`gpio_out_sel[12]`. Per the ESP32 Technical Reference Manual section
4.11, Table 4-3:
71 .. 78 → LEDC HS channels 0..7
79 .. 86 → LEDC LS channels 0..7
The legacy worker code at esp32_worker.py:426 used the off-by-one
range `72 <= signal <= 87` with `ledc_ch = signal - 72`. The mistake
masked itself for single-servo projects because the 0x5000 duty
callback's channel index was internally consistent with the bogus
math, so the duty STILL reached the correctly-routed pin (just
labelled wrong). The new SignalRouter unit tests caught the
discrepancy the moment two servos drove distinct channels: signal
71 (HS_CH0, gpio 13) was REJECTED by the off-by-one filter and
signal 72 (HS_CH1, gpio 12) was misclassified as channel 0.
When I ported the legacy range into `esp32_signals.SIG_LEDC_HS_CH0_OUT_IDX`
the bug came along for the ride. Fix both modules:
* `backend/app/services/esp32_signals.py`: HS 71-78, LS 79-86.
* `frontend/src/simulation/esp32-signals.ts`: mirror.
* tests updated; 20 backend + 23 frontend pass.
After deploy the user's two servos will resolve to their declared
pins:
ch=0 duty=X% gpio=13 (servoPan, was wrongly emitting gpio=12)
ch=1 duty=X% gpio=12 (servoTilt, was wrongly emitting gpio=-1)
This is also why the multi-servo blink "patch" in commit 77bf897
appeared to help: with both pins ALIASED to the same channel via
the off-by-one, the broadcast fallback was the only thing producing
ANY movement on the second servo at all.
The esp32_worker.py subprocess is launched via `python <abs_path>`
and runs with a sys.path that does NOT include the backend/ package
root, so `from app.services.signal_router import SignalRouter`
raised ModuleNotFoundError at worker startup. The worker exited
with code 1 before QEMU even loaded, and the frontend surfaced the
generic "ESP32 crash detected — cache error" banner.
Mirror the existing esp32_flash_image fallback pattern (already in
this same file): try the package import first, fall back to
importlib.spec_from_file_location with the sibling .py path, then
publish the resulting module under its bare name in sys.modules so
typing references continue to work.
Verified: a synthetic test that strips backend/ from sys.path can
still construct a SignalRouter via the fallback. 20 unit tests in
test_signal_router.py still pass.
Replaces the per-peripheral ad-hoc `_ledc_gpio_map` cache with a
proper signal-routing abstraction that mirrors the ESP32 SoC's
IO_MUX + GPIO Matrix exactly. Same idea as real silicon: signal
sources (LEDC channels, RMT, MCPWM, ...) → 40-entry routing table
→ GPIO pins.
Motivation (from user bug report in
velxio.dev/project/5218f9e3-136d-43b3-bba1-6cebde21e1a4): two
ESP32 servos on a solar-tracker visibly oscillated between two
positions instead of moving smoothly when the user changed LDR
sliders. Commit 77bf897 patched it (per-channel gpio memo +
broadcast guard) but the user requested a proper hardware-fidel
architecture, not patches.
Backend:
* `app/services/signal_router.py` — SignalRouter class. Forward
index (gpio → signal_id) + reverse index (signal_id → set of
gpios). `replace_snapshot()` returns the diff for the polling-
fallback path; future C plugin hook becomes a push without
touching this code.
* `app/services/esp32_signals.py` — Signal id constants from
ESP32 TRM (LEDC HS 72-79, LS 80-87) + `ledc_signal_for_channel()`
helper.
* `app/services/esp32_worker.py` — `_ledc_gpio_map` is gone;
`_refresh_ledc_gpio_map` replaced by `_refresh_signal_routing`
which emits `gpio_routing {gpio, signal_id}` events on diff.
The 0x5000 LEDC callback and the LEDC poll thread now emit
`ledc_duty {channel, duty_pct}` (canonical, no gpio) alongside
the legacy `ledc_update {channel, duty, gpio}` for back-compat
during rollout.
Frontend:
* `simulation/SignalRouter.ts` — 1-to-1 TS mirror of the Python
class. Same forward + reverse index; same `pinsForSignal` /
`updateRouting` / `clearRouting` API.
* `simulation/esp32-signals.ts` — Signal id constants, mirror
of the Python module.
* `simulation/Esp32Bridge.ts` — new `onLedcDuty`, `onGpioRouting`,
`onGpioRoutingClear` callbacks; handlers for the new event types.
* `store/useSimulatorStore.ts` — `makeLedcDutyHandler` looks up
pins via `router.pinsForSignal(ledcSignalForChannel(channel))`
and dispatches per pin. `makeGpioRoutingHandler` /
`makeGpioRoutingClearHandler` keep the mirror in sync. Per-board
`signalRouterMap` parallels `pinManagerMap` in lifecycle.
`makeLedcUpdateHandler` (and its memo workaround from 77bf897)
stays wired for back-compat during rollout; removed in a
follow-up commit once prod is verified stable on the new path.
Tests:
* `test/backend/unit/test_signal_router.py` (20 tests) covers
update/clear semantics, idempotency, multi-pin routing,
snapshot diff, channel↔signal-id helpers, and the multi-servo
regression scenario.
* `frontend/src/__tests__/SignalRouter.test.ts` (17 tests) is the
mirror — same scenarios on the TS side.
* `frontend/src/__tests__/esp32-multi-servo-gpio-matrix.test.ts`
(6 tests) drives the end-to-end SignalRouter handler pipeline,
asserts that two servos on GPIO 13/12 via LEDC channels 0/1
move independently (no mirroring), that re-routing carries
cleanly, and — critically — that `PinManager.broadcastPwm` is
never called.
Totals: +700 LOC, 1876 frontend tests pass (was 1853), 278 backend
unit tests pass (was 259).
Docs: ESP32_EMULATION.md §9.2 rewritten with the new architecture
diagram + a runbook for adding future peripherals through the
SignalRouter.
The C plugin hook in qemu-lcgamboa that would push gpio_out_sel
writes synchronously (eliminating the polling race window entirely)
is the next step — kept as a follow-up because the polling-fallback
path here already resolves the routing before each duty event
fires, so the bug is fixed end-to-end. The plugin work removes the
race condition fundamentally.
User-reported bug: Pi 3 simulator showed boot output but keyboard
input was ignored — the shell was effectively read-only.
Root cause: velxio-init's bash redirect was `</dev/console
>/dev/console`. From userspace, /dev/console is write-only — it is
the kernel's printk target and accepts writes (so we saw boot output
fine) but reads return EOF / block forever. Bash never saw a
keystroke and the user couldn't type.
Fix: read AND write through /dev/ttyAMA1. The 12 s devtmpfs-wait
already in velxio-init guarantees the device node exists by the
time the shell-respawn loop runs. setsid -c still gives bash a
controlling terminal so PS1, job control, and Ctrl-C all work.
Manifest version bumped to 2026-04-21+ttyAMA1; sidecar SHA check
invalidates the cached SD on every velxio backend so the fix lands
without an operator dance.
The previous velxio-init was racing devtmpfs population: its bash
redirect '</dev/ttyAMA1 >/dev/ttyAMA1' fired before the kernel had
enumerated the PL011 driver and populated the device node, so PID 1's
fd 0/1/2 redirect failed and the `while true` loop spun on
"No such file or directory" forever.
Two fixes baked into the SD image:
1. Wait up to 12 s for /dev/ttyAMA1 to appear (200 ms poll × 60).
On real bare-metal Pi the node is there at init time, but
under QEMU emulation the PL011 probe races.
2. Exec the shell with </dev/console >/dev/console — /dev/console is
set up by the kernel (no race) and points at the last `console=`
arg from the cmdline, which is ttyAMA1. Also wrap in `setsid -c`
so bash gets a controlling terminal and behaves interactively.
Verified end-to-end with a live QEMU boot against the patched .img:
shell prompt `root@raspberrypi:/#` appears within ~50 s wall (most
of that is the kernel waiting on the second SD slot mmc1 timeout
twice = 20 s).
Pi 3 simulator boot through Pi OS systemd graph was unworkable inside
QEMU's raspi3b emulation:
* The PL011 UART at 0x3f201000 enumerates as ttyAMA1 (not ttyAMA0 —
the mini-UART at 0x3f215040 takes ttyAMA0 and fails to probe under
QEMU). After ~9 s of kernel time the boot effectively went silent
on the serial: earlycon was disabled by the normal console init
and the IRQ-driven serial driver loses TX under QEMU's emulation.
* Even with `keep_bootcon`, systemd dependency graph took 2-3 min to
walk inside emulation (network waits, tmpfiles, journald,
hostname/machine-id randomness). Masking 9 boot-blocking units
helped but didn't fix the silent-after-9s problem.
Solution: skip systemd. The SD image is now baked with
`/usr/local/sbin/velxio-init` (a 30-line bash script) and the kernel
cmdline points init= at it. velxio-init mounts /proc /sys /dev /pts
/run /tmp, sets hostname, then loops a passwordless `/bin/bash
--login </dev/ttyAMA1 >/dev/ttyAMA1`. User sees the prompt within
~10 s of clicking Run; Ctrl-D respawns a fresh session.
Cmdline additions:
- `keep_bootcon` — keep earlycon alive after the regular console
registers, so kernel printk continues to reach ttyAMA1.
- `console=ttyAMA1,115200` — the correct PL011, not ttyAMA0.
- `init=/usr/local/sbin/velxio-init` — bypass systemd entirely.
Python, GPIO shim, apt, mount, etc. all work — they don't need
systemd as PID 1, just a populated rootfs + mounted pseudo-fs.
Manifest version bumped to 2026-04-21+velxio-init. Same byte size,
different SHA, so the sidecar-based cache invalidator forces a
re-fetch on every velxio backend the next time it starts.
Boot from cold to root prompt was 2-3 min because Pi OS Trixie waits
on a handful of services that timeout instead of completing:
- systemd-networkd-wait-online (60s default)
- NetworkManager-wait-online (30s default)
- wpa_supplicant + dhcpcd5 (no usable interfaces)
- raspi-config / firstboot / userconfig (no point in QEMU)
The SD image was re-baked through scripts/configure-pi3-autologin.sh
with all of them masked (the script grew a `mask_unit` helper that
symlinks each unit to /dev/null inside the rootfs). Login prompt now
appears in ~30s wall.
New manifest version 2026-04-21+autologin+fastboot — same byte count
as the previous build (still 5.4 GiB raw) but a different SHA so the
sidecar-based cache invalidation forces every container to refetch.
Two more defects making Pi 3 boot silently:
1. The kernel8.img that ships in the Pi OS armhf boot partition is a
gzip-compressed PE-COFF Image (first 4 bytes 0x1f8b0800). QEMU's
`-kernel` does NOT auto-decompress; it tries to execute the gzip
header as ARM code and the CPU faults immediately. Result: zero
bytes on ttyAMA0, simulator looks dead. Switch the asset_id to a
pre-decompressed kernel (24 MiB raw vs 9.7 MiB gzipped) so QEMU
gets a valid Image to boot.
2. Even with a real kernel, the original cmdline `console=ttyAMA0`
alone wasn't enough — the kernel can't initialise the BCM2837
PL011 UART early enough for `printk` to reach the serial console
under QEMU's bare-metal boot (no Pi firmware to set it up
beforehand). Adding `earlycon=pl011,mmio32,0x3f201000` makes the
kernel program the UART itself in the early boot path.
Verified: boot output starts streaming within 100 ms of QEMU
launch instead of never.
The cmdline also locks the baud rate at 115200 to match the agetty
drop-in created by scripts/configure-pi3-autologin.sh.
User report: clicked Pi 3 board → nothing visible happens. Three
defects, all on the same path:
1. The kernel cmdline carried over from the original pre-OSS-split
code: `quiet init=/bin/sh`. Result: kernel boot messages
suppressed, then dropped straight to bare /bin/sh with no PS1 so
the user sees an empty serial. Removed both. The kernel cmdline
is now just `console=ttyAMA0 root=/dev/mmcblk0p2 rootwait rw
dwc_otg.lpm_enable=0`, which lets systemd start a real
serial-getty@ttyAMA0.service.
2. Pi OS Trixie armhf since Bookworm ships without a default user
(no more pi/raspberry). With cmdline #1 fixed, the user would
land at a login prompt and be stuck. Fix: pre-bake a systemd
drop-in at /etc/systemd/system/serial-getty@ttyAMA0.service.d/
autologin.conf that uses `agetty --autologin root` so the serial
console drops to a root shell on first prompt. The browser
canvas IS the authentication boundary; the SD image is mounted
RO via a qcow2 overlay so per-session edits don't persist.
Edit happens in velxio-prod/scripts/configure-pi3-autologin.sh
(to follow in a separate commit).
3. Architectural: the original cache-hit probe was size-only.
Today's SD image rebake produced a file with identical byte count
but different SHA256 — the cache served stale content for every
request even after a manifest bump. Fix: write a sidecar
`<file>.sha256` after every successful materialise and trust it
on subsequent probes. Manifest SHA bumps invalidate the cache
regardless of size. Two regression tests guard this:
- test_provider_sidecar_invalidates_on_sha_mismatch
- test_provider_missing_sidecar_treats_file_as_invalid
Manifest bumped to version "2026-04-21+autologin" for the SD image
(kernel + DTB unchanged, still 2026-04-21).
Pi 3 simulation had been broken since at least April 2026 (51
fail-events / 24h per docs/PI3_EMULATION_BROKEN.md). Two distinct
defects compounded:
1. qemu_manager.py hard-coded paths for kernel8.img, a device-tree
blob (under a DOS 8.3 short name!), and a 5.4 GiB Raspberry Pi OS
SD image — none of which shipped in the repo or were pulled at
image build.
2. qemu-system-arm + qemu-utils were missing from the Docker image
entirely, so even with the boot files in place QEMU couldn't
launch. Add both to Dockerfile.standalone (~200 MB).
The architecture fix is a new `app.services.boot_images` module:
* Manifest-driven (boot_images/manifest.json, versioned in repo,
declares SHA256 + size for each file, supports an optional
`compressed.{encoding,sha256,size_bytes}` block for assets shipped
as .zst).
* `BootImageProvider` materialises files lazily, atomically (temp +
rename), verifies SHA256 pre- AND post-decompression, caches under
/var/cache/velxio/boot-images, serialises concurrent get() calls
per image set via asyncio.Lock.
* `AssetDownloader` Protocol with two impls:
- `LicenseGatedDownloader` — same flow ESP32 / RISC-V QEMU libs
use (VELXIO_BINARY_BASE_URL + VELXIO_LICENSE_KEY).
- `LocalDirectoryDownloader` — for tests + in-prod use where the
licence-module storage is already on the same filesystem (saves
the loopback HTTP roundtrip on a 1.4 GiB blob).
* `build_downloader_from_env()` picks one — local-dir wins if both
sets of env vars are present, so the prod box short-circuits to
direct disk reads automatically.
* Lifespan hook in qemu_manager.py pre-warms the cache on container
boot so first-time user requests don't pay the 30-60 s download
+ decompress latency.
Adding a future board kind (Pi 4 / Pi 5) is now: upload assets via
upload-binary.sh, append an entry to manifest.json, register a
lifespan pre-warm in the new board's service module. Zero edits to
provider.py / downloader.py.
Manifest entries for raspberry-pi-3:
kernel8.img 9 695 883 bytes (uncompressed)
bcm2710-rpi-3-b.dtb 34 687 bytes (uncompressed)
raspios-trixie-armhf.img 5 729 419 264 bytes raw
/ 1 488 002 803 bytes .zst on wire (zstd -19)
source: 2026-04-21 build from raspberrypi.com
Tests: 21 new unit tests covering manifest parsing, integrity
helpers, both downloaders, and the provider's idempotent /
concurrent / integrity / decompression / warmup paths. In-process
FakeDownloader keeps the suite under 1 s and httpx-free.
Docs: new docs/BOOT_IMAGES.md describes the architecture, on-disk
layout, named-volume operation, and the procedure for adding a new
image set.
After Phase 4 of the OSS / pro split, the OSS code base imports zero
auth/DB modules (verified with grep across backend/app/). But the
requirements.txt + config.py + .env.example + docs still listed
SQLAlchemy, aiosqlite, JWT/bcrypt, OAuth, SECRET_KEY etc. as if they
were live. Self-hosters running `pip install -r requirements.txt`
were pulling ~30 MB of packages the code never imports.
Changes:
* backend/requirements.txt — drop sqlalchemy, greenlet, aiosqlite,
python-jose, passlib[bcrypt], bcrypt, authlib, email-validator,
python-multipart. Keep fastapi, uvicorn, websockets, pydantic,
pydantic-settings, httpx, mcp, esptool, wasmtime — everything OSS
actually uses.
* backend/app/core/config.py — Settings reduced to FRONTEND_URL only.
Comment explains the overlay path that adds the rest at Docker
build time.
* backend/.env.example — same trim: only FRONTEND_URL, with a comment
explaining why this file is almost empty.
* README.md — "Auth & Project Persistence" section rewritten to
describe .vlx export/import. Env-var table reduced to a single row.
Stack table updated: no SQLAlchemy, no JWT, persistence = .vlx
files.
* CLAUDE.md — intro line updated (Auth: None, persistence: .vlx).
Key-file-locations rewritten to list the OSS-stateless backend +
the new lib/proRoutes / proSession / proSaveAction seams, with an
explicit "removed in the split" note pointing to velxio-prod.
Stores section drops useAuthStore (overlay-only now). Backend
gotchas drop the bcrypt + email-validator + model-import notes.
Implemented-features list replaces "Auth + URL persistence + user
profile" with portable .vlx export/import.
* docs/ESP32_EMULATION.md — two `docker run` examples dropped the
`-e SECRET_KEY=...` arg (no longer needed).
OSS build verified end-to-end (285 SEO pages prerender, 20 stateless
routes, zero sqlalchemy imports).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Without the type annotation, FastAPI treats `request` as a Query
parameter and bubbles it up to every endpoint that uses
`Depends(get_current_user_id)`. Result: POST /api/compile/start
and POST /api/compile/ both returned 422
{"loc":["query","request"],"msg":"Field required"} on every call
the frontend made — compile was fully broken in production.
The frontend then caught the 422 axios error and surfaced
response.data as a CompileResult, which had no success/stdout/
stderr/error fields, so the editor's CompilationConsole rendered
only the fallback "✕ Compilation failed" line with no detail.
Annotating `request: Request` is the standard FastAPI pattern;
the framework injects the raw HTTPRequest and no longer treats
it as a query parameter.
Phase 2 of the OSS / pro split. The hook seams introduced in Phase 1
let stateless routes (compile, libraries, simulation, iot_gateway)
run without the auth/DB stack importable. Now we actually delete the
stack:
app/api/routes/auth.py
app/api/routes/projects.py
app/api/routes/admin.py
app/api/routes/metrics.py
app/models/{user,project,usage_event,password_reset_token}.py
app/schemas/{auth,admin,project}.py
app/core/{dependencies,security}.py
app/database/session.py
app/services/{metrics,odoo_mail,project_files}.py
app/utils/{geo,slug,boards}.py
Private deployments (velxio.dev) get the same modules back via the
velxio-prod overlay: pro/backend/app/api/routes/auth.py etc. are
COPYed onto /app/... at container build time, and register_pro()
includes their routers + registers the lifespan/metrics/auth hooks.
main.py shrank back to the stateless router includes + a single
`run_lifespan_startup()` call. The Phase-1 try-import block that wired
record_compile / get_current_user_id from upstream is gone — those
adapters live in pro now.
Verification:
OSS only: 20 routes (compile, libraries, simulation, gateway).
OSS + pro: 94 routes — identical to pre-refactor velxio.dev.
Net change: -2400 lines from OSS, all of which moved to velxio-prod's
overlay. Self-hosted OSS users lose accounts + project persistence;
the Phase 4 .vlx export/import gives them a portable replacement.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First phase of the OSS / pro split. Goal: open the seams so the auth/DB/admin
stack can move into the private overlay (Phase 2-3) without the routes that
stay in OSS (compile, libraries, simulation, iot_gateway) having to know.
Backend
-------
* New app/core/hooks.py — registry for record_compile, get_current_user_id,
and lifespan startup tasks. Each hook is a no-op by default; overlays
call register_* in register_pro(app) to plug in a real implementation.
* compile.py now imports only from app.core.hooks. Drops the direct deps on
app.core.dependencies, app.database.session, app.models.user, and
app.services.metrics. Route signatures use `Depends(get_current_user_id)`
instead of `Depends(get_current_user)`; the metric helper passes user_id
through rather than a User instance.
* compile_chip.py drops the unused _current_user Depends entirely.
* main.py wraps the auth/DB stack import in try/except. When it succeeds
(today's behavior on velxio.dev), an adapter bridges record_compile and
get_current_user_id to the existing app.services.metrics + dependencies,
and the create_all + ALTER TABLE migration block runs via a registered
lifespan_startup hook. When it fails (the post-Phase-2 OSS image), main
logs "running stateless" and skips registering anything — the routes
still load and behave as no-ops for metrics + always-anonymous for auth.
Frontend
--------
* useAutoSaveProject becomes a skeleton: one useState + one useEffect that
delegates to an installed AutoSaveImpl. installAutoSaveImpl() replaces
the impl without changing hook count, so React's rules-of-hooks stay
satisfied even after the impl moves out of OSS.
* New hooks/autoSaveImpl.ts holds the original logic (debouncing, dirty
detection, owner eligibility, fetch keepalive on unload), refactored to
emit() instead of useState. It self-registers at module load; main.tsx
imports it for the side effect.
* AppHeader wraps the entire user-vs-login UI in a data-velxio-slot
="header-auth" boundary. Today the OSS UI still renders inside the slot
— the overlay can portal-inject additional items now, and in Phase 3
the slot becomes the sole owner of header auth UX.
Behavior is identical on velxio.dev (pro overlay imports everything
successfully, every adapter wires up). The change is purely structural:
deleting the auth/DB modules tomorrow no longer crashes OSS at import.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
arduino-esp32 v2.0.17's libraries/WiFiClientSecure/src/ssl_client.cpp:23
wraps its entire body in:
#if !defined(MBEDTLS_KEY_EXCHANGE__SOME__PSK_ENABLED) \
&& !defined(MBEDTLS_KEY_EXCHANGE_SOME_PSK_ENABLED)
# warning "Please call idf.py menuconfig ..."
#else
ssl_init / start_ssl_client / stop_ssl_socket /
send_ssl_data / get_ssl_receive / data_to_read
#endif
Our esp-idf-template/sdkconfig.defaults did not enable any PSK key-exchange
mode, so MBEDTLS_KEY_EXCHANGE_SOME_PSK_ENABLED was never auto-set by
mbedtls and ssl_client.cpp compiled to an empty translation unit. The
companion WiFiClientSecure.cpp still compiled and ended up in
libarduino-esp32.a with dangling references, breaking the link of every
sketch that pulls in HTTPClient or WiFiClientSecure (directly or
transitively).
Reproduced against the user's WiFi + HTTPClient example.com sketch on the
prod server and again locally with ESP-IDF v4.4.7 + arduino-esp32 v2.0.17;
the prebuilt sdkconfig that ships with arduino-esp32 itself sets both
flags, so we just align with that.
After the fix the same sketch links cleanly:
velxio-sketch.bin binary size 0xbf470 bytes ... 25% free
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the register-then-immediately-forgot race for good. Two parallel
calls to /velxio/api/send-welcome and /velxio/api/send-password-reset
were hitting Odoo's REPEATABLE-READ snapshot timing: both workers took
their snapshot BEFORE either had committed the partner row, so each
ran its own INSERT and the second blew up on the velxio_user_id
unique constraint — costing one of the two emails.
Add a new sync_partner() helper that POSTs to a new Odoo route
/velxio/api/upsert-partner (lives in velxio_subscription) which only
upserts the partner — no mail, no subscription, fast. Register and
forgot-password await this BEFORE firing the async welcome / reset
tasks. Each user's flow is sequential at the Velxio HTTP layer, so
the snapshot race disappears.
sync_partner() reuses the existing _post() error-swallowing pattern.
If Odoo is down, sync_partner returns None and the await is a no-op
— the user still registers / gets the generic 200, the welcome /
reset endpoints retain their defensive upsert as a fallback path.
David Montero