arduino-esp32 3.x ledcWrite takes a PIN and looks up the attached channel
internally. arduino-esp32 2.x (the toolchain version we pin) takes a
CHANNEL. The velxio_compat.h shim already aliased the 3.x-only
ledcAttach onto ledcSetup+ledcAttachPin so 3.x sketches would compile,
but ledcWrite still mapped 1:1 — so a call like
#define R_PIN 16
ledcAttach(R_PIN, 5000, 8); // shim → channel 0 attached to pin 16
ledcWrite(R_PIN, 128); // ★ writes to "channel 16" (invalid)
silently wrote to LEDC channel 16, which doesn't exist (valid range
0-15). The hardware duty register never changed, qemu-lcgamboa never
emitted a `ledc_duty` event, and the RGB LED stayed dark even though
the firmware ran cleanly and the wires looked right. Verified end-to-end
with examples/esp32-pwm-led-rgb: gpio_change events fired at boot, no
ledc_duty events fired, ledRed/ledGreen/ledBlue all stayed at 0.
Fix: maintain a 40-entry pin→channel table populated by both ledcAttach
variants. Replace ledcWrite with a macro that calls a helper checking
the table first; if the value isn't a known pin we pass it through as a
channel, preserving 2.x channel-style call sites.
Macro/function name collision is sidestepped with the standard
parenthesizing trick — `(ledcWrite)(channel, duty)` doesn't expand the
function-like macro because the token isn't followed by `(`.
Verified live on velxio.dev/example/esp32-pwm-led-rgb after hot-copying
the new header into the velxio-app container: ledRed/ledGreen/ledBlue
now cycle through the full HSV wheel as expected (samples: (255,41,0),
(41,255,0), (0,41,255), (232,255,0), …).
Single-file sketch only — the table is `static` (internal linkage) and
ledcAttach + ledcWrite live in the header. Multi-file sketches that
attach in file A and write in file B would each see their own table.
Acceptable for now since arduino-esp32 sketches are nearly always
single-file; revisit when we bump the toolchain to 3.x and can drop the
shim entirely.
ATTinyCore >=1.5.0 declares ATTinyCore:micronucleus@2.5-azd1b as a tool
dependency, hosted at https://azduino.com/bin/micronucleus/. That host
has been unreachable (connection refused) for extended periods, causing
every ATtiny85 compile to fail at the core-install step with:
Download failed: performing HEAD request: ... dial tcp ...: connection refused
Failed to install required core: ATTinyCore:avr
micronucleus is only used for USB upload — never for compilation — but
arduino-cli refuses to install a core whose tool deps cannot fetch.
Pin to 1.4.1, the last release whose micronucleus binary is hosted on
github.com (digistump release, reachable). The FQBN clock options we
ship (clock=16pll on attinyx5, etc.) are unchanged across 1.4.x.
- backend/app/services/arduino_cli.py: new CORE_INSTALL_VERSIONS map
consulted by ensure_core_for_board so the runtime auto-install
passes "ATTinyCore:avr@1.4.1" instead of unversioned latest.
- backend/Dockerfile and docker/entrypoint.sh: same pin so a fresh
image bakes 1.4.1 in and never hits the runtime fallback path.
Existing regression tests in test/backend/unit/test_arduino_cli_attinycore.py
still pass (they assert presence, not version).
Two CI failures landed after PR #196 (esp32-gpio-matrix-cb-callback)
merged. Both are independent and fixed here together.
1) **Backend E2E: ESP32 hangs at bootloader handoff.**
PR #196 added picsimlab_gpio_matrix_cb which fires on QEMU's
iothread. The handler did `_emit({...})` for every routing
change — and the ESP-IDF bootloader writes to gpio_out_sel
*hundreds* of times during early boot (each peripheral init
configures its matrix slot). Each emit acquires _stdout_lock
and writes to the worker→manager pipe. If the manager drains
even briefly slow, the pipe fills, write blocks, and the
iothread stalls — symptom: ESP32 reports `entry 0x400805e4`
then no Arduino setup() output for 75 s.
Fix: the iothread callback now ONLY mutates the SignalRouter
snapshot. It never emits. The 10 Hz poll thread
(_refresh_signal_routing) stays as the sole emitter, so the
wire-format event stream is unchanged. Benefit of having the
callback over poll-only is reduced worst-case routing-emit
latency (next poll tick vs up to 100 ms) and a warmer
snapshot dict for cheaper poll diffs.
2) **Frontend Tests: Node OOM at end of suite.**
117 test files run in one forks-pool worker. Several lazy-load
the ngspice emscripten module (~30 MB), the MixedModeScheduler
singleton, and other heavy modules whose dispose hooks aren't
reached because singletons leak across files. Cumulative heap
pressure exceeds Node's 4 GB default; the worker hits "Ineffective
mark-compacts near heap limit" AFTER all 1881 tests pass and
the OOM kill is reported by vitest as "Worker exited unexpectedly
/ Timeout terminating forks worker". This is not a real test
failure — every individual test passes.
Quick fix: pass NODE_OPTIONS=--max-old-space-size=8192 to the
`npm test` step. Long-term, the singletons should add dispose
hooks that test fixtures call in afterAll(), or the suite
should shard into multiple `vitest run --shard` invocations.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires the new synchronous GPIO Matrix callback exposed by
libqemu-{xtensa,riscv32} 1.1.0 (lcgamboa/qemu commit e178ff5).
Whenever the firmware writes GPIO_FUNCx_OUT_SEL_CFG_REG, the C
plugin now fires picsimlab_gpio_matrix_cb(gpio, signal_id) inline.
The handler:
- Treats signal_id == 0x100 or 0 as "matrix routing cleared" and
emits gpio_routing_clear.
- For LEDC HS/LS range signals (the only ones the frontend
SignalRouter currently consumes), updates the mirror and emits
gpio_routing.
- Drops other signals — the mirror does not need to track them
yet, and emitting them would only fatten WS frames.
Backwards compat:
- Older libqemu (<1.1.0) doesn't expose the new field; the
picsimlab_gpio_matrix_cb placeholder runs (no-op) and the
100 ms _refresh_signal_routing() poll thread continues to feed
the mirror. WS event shape is identical either way.
Burn-in: keeping the poll thread active in parallel with the
callback for now. Once telemetry confirms parity (per phase 4 doc
in velxio-prod/project/esp32-gpio-matrix-cb/), the poll thread
gets retired in a follow-up commit.
Adds a third format to /api/compile-rom: `c` (C source compiled by SDCC
to Z80 bytes). Same chip-program flow as 8080/Z80 asm — write C in a
project file, click Compile, click Run.
Backend:
- backend/app/services/c_compile.py — async SDCC wrapper. Locates the
sdcc binary on PATH (or via SDCC env var, or common Windows install
paths) and shells out with target=mz80 + --code-loc 0x100 --data-loc
0x8000. Parses the resulting Intel HEX into raw ROM bytes. Pure 8080
is rejected with a clear error (SDCC has no 8080 backend; Z80 ROMs
also run on the i8080-cpu chip if you avoid Z80-only ops).
- rom_compile.py: compile_rom is now async; the new c branch delegates
to c_compile. compile_rom_endpoint awaits it.
Frontend:
- romCompileService: RomFormat gains 'c'; formatForFile maps .c/.cpp to
'c'. isChipProgramFile intentionally still excludes .c — disambiguation
happens at the EditorToolbar level.
- EditorToolbar: the chip-program path also fires when a custom-chip
has programFile === activeFile.name (regardless of extension). That
lets .c files route to /api/compile-rom (SDCC) when bound to a CPU
chip, while .c files NOT bound to any chip continue to route to
arduino-cli as before.
Docker:
- Dockerfile.standalone adds `sdcc` to the apt-get install list, so the
prod image ships with SDCC out of the box.
Example:
- /examples/z80-led-chaser-c — z80-cpu chip + chaser.c (a Larson
scanner written in C with __at() MMIO definitions). Compiles cleanly
with SDCC's --code-loc 0x100 default crt0.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the Zilog Z80 to the programmable-retro-CPU lineup. Same compile-rom
flow that landed for the 8080 in PR #189: write Z80 asm in a project
file, click Compile (backend assembles via in-tree two-pass asm-z80),
click Run, the chip emulator boots from the resulting ROM bytes.
Backend:
- backend/app/services/asmz80.py — two-pass Z80 assembler covering the
practical demo subset: LD r,n / r,r' / rp,nn / (nn),A / A,(nn) +
ALU r/n + INC/DEC + JP/JR/DJNZ/CALL/RET + PUSH/POP + IN/OUT +
EX/EXX + LDIR/LDDR/IM/NEG + RLCA/RRCA/RLA/RRA + the simple
ED-prefix variants. Not yet: CB-prefix bit ops, DD/FD index ops.
- rom_compile.py routes target=z80 through the new assembler.
Chip:
- frontend/src/components/customChips/examples/intel/z80-cpu.{c,chip.json}
Generated by scripts/make-z80-cpu.py from the existing z80.c emulator
(same clean-room implementation that passes ZEXDOC end-to-end). The
external pin/bus protocol is replaced with internal RAM + ROM + MMIO
for LED/BTN/UART. 35 KB WASM.
Example:
- /examples/z80-larson-scanner — Knight-Rider-style walking LED.
Demonstrates JR/DJNZ/RLCA which the 8080 can't run.
Plus a small Z80 smoke-test asm under scripts/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a new way to use the retro CPU chips: write your program in a
project file (.s / .asm / .hex / .bin), click Compile, click Run, and
the same chip emulates whatever you wrote. Same chip + different ROMs =
mini PC, calculator, LED demo, Kill-the-Bit game, etc.
SDK:
- velxio-chip.h gets two new host imports:
uint32_t vx_rom_size(void);
void vx_rom_read(uint32_t off, uint8_t* dst, uint32_t len);
CPU-emulator chips call these in chip_setup to pull their program out
of the host's romBytes property.
Frontend runtime:
- ChipRuntime accepts opts.romBytes (Uint8Array) and exposes the new
imports, copying bytes into chip memory on vx_rom_read.
- CustomChipPart pulls component.properties.romBytes (base64) and passes
it through.
- Component registry declares three new custom-chip properties:
romBytes (base64), programFile (matching project filename), and
programTarget (cpu name).
New programmable bundled chip:
- frontend/src/components/customChips/examples/intel/i8080-cpu.{c,chip.json}
Same clean-room 8080 emulator as i8080-repl/i8080-counter, but ROM is
loaded externally via vx_rom_*. Has 8 LEDs, 8 buttons, UART, 16 KB RAM,
32 KB of external ROM.
Backend:
- New /api/compile-rom endpoint and rom_compile service that turns
chip-program source into ROM bytes. 8080 ASM is assembled by the
in-tree two-pass assembler (moved to backend/app/services/asm8080.py).
Intel HEX records are parsed; raw .bin is passed through. Future targets
(z80, 8086, 4004) are scaffolded but not wired yet.
EditorToolbar:
- Compile button detects when the active file is .s/.asm/.hex/.bin and
routes to compile-rom instead of arduino-cli. The compiled bytes are
injected into every custom-chip on the canvas whose programFile property
matches the active filename (or is empty).
Example:
- /examples/i8080-killbits loads Dean McDaniel's 1975 Kill-the-Bit on
the programmable i8080-cpu chip. killbits.s is shipped as a project
file alongside sketch.ino; the user clicks Compile then Run and the
LED walks across 8 outputs, buttons kill it.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The SignalRouter path has been in prod through Phase 2.5 / Phase 3.3
deploys without regressions, so the temporary fallback shipped in
commit 77bf897 can come out. Closes#101.
Backend (esp32_worker.py + esp32_lib_manager.py):
- Stop emitting `ledc_update` from the 0x5000 LEDC callback and from
the polling thread. Only `ledc_duty` (channel + duty_pct) and the
GPIO matrix routing events ship now.
- Drop the channel→gpio reverse-lookup that fed the legacy event.
Frontend:
- Delete `PinManager.broadcastPwm` and `PinManager.pwmListenerPinCount`.
- Delete `makeLedcUpdateHandler` + its `channelGpioMemo`.
- Delete `Esp32Bridge.onLedcUpdate` field + the `case 'ledc_update':`
message handler + the `LedcUpdate` type.
- Strip `this.onLedcUpdate = null` from 14 test mocks.
- Rewrite the `does not call broadcastPwm` guard in
esp32-multi-servo-gpio-matrix.test.ts to assert the method itself
no longer exists on PinManager (stronger regression guard than the
spy version, and doesn't need vi).
- Remove the `PinManager.broadcastPwm fallback` describe block from
esp32-servo-pot.test.ts — every test in it exercised the deleted
fallback path.
Docs (ESP32_EMULATION.md):
- Replace `ledc_update` rows in the events / implementation tables
with the SignalRouter trio (`ledc_duty`, `gpio_routing`,
`gpio_routing_clear`).
- Update the visual flow diagram + the "why this matters" paragraph
to past-tense the broadcastPwm bug.
Tests: 1886 frontend tests pass (the previously-failing
board-kinds-coverage test that needed the new Pi Zero/1/2 kinds is
also green). Backend unit suite: 279 pass, the 11 espidf_real_paths
prereq failures are environment-dependent (need arduino-cli libs in
the local shell) and unrelated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
End-to-end pipeline fixes uncovered while auditing the /examples gallery.
Each bug shipped past green unit + snapshot tests because none of those run
firmware + render LEDs. Added scripts/visual-led-test.mjs as a CDP-driven
visual harness that loads each example, runs the simulator, samples
`wokwi-led.brightness`, and asserts toggle / gradient / initial-off
invariants — exits non-zero on any regression.
Frontend simulator
- PinManager.updatePort: new optional ddrMask param. A pin is added to
`outputPins` only if the DDR bit is set, so the PORTx write that
enables INPUT_PULLUP (DDR=0, PORT=1) no longer falsely marks the pin
as MCU output. AVRSimulator now reads DDRB/C/D (0x24/0x27/0x2A on
Uno/Nano, 0x37 on ATtiny85, per-port table on Mega) and forwards it.
- AVRSimulator: pass DDR mask alongside every port-listener fire.
- BasicParts pushbutton{,-6mm}: seed pin HIGH in attachEvents so
`digitalRead()` returns HIGH while idle. avr8js doesn't auto-simulate
INPUT_PULLUP — without this the firmware reads LOW from boot and
thinks the button is permanently pressed (the "LED is always on,
pressing does nothing" UX bug).
- connectMcuEdgesToService: suppress synthetic digital edges on pins
with active PWM, AND subscribe to onPwmChange to re-tick the netlist
on duty changes. Fade-LED now produces a true gradient (6 distinct
brightness levels across a fade cycle) instead of a binary 0/full
toggle.
- CircuitSimulationService.handleMcuEdge: replace single-slot
pendingMcuEdge with a per-pin Map. Multiple pins toggling during the
same in-flight tick used to overwrite each other; now every pin's
most-recent edge replays after the tick. Fixes Traffic-Light RED→
YELLOW→GREEN sequencing.
- NetlistBuilder: new sanitizeSpiceId() helper replaces hyphens with
underscores in V-source names. ngspice's interactive `alter` command
treats `-` as an operator and silently no-ops on hyphenated source
names, so mid-simulation MCU pin transitions stopped propagating
after the first solve. MixedModeScheduler.onMcuPinChange and
CircuitSimulationService self-heal use the same sanitizer so names
stay consistent across emit/alter/lookup. Also added a regex-based
fallback in step 2 so any board pin matching `GND.\d+` canonicalises
to net "0" — ESP32-C3 dev kits expose up to 10 GND pins and the
per-board `groundPinNames` list missed several, leaving wires
floating instead of grounded.
- collectPinStates: emit V-sources only for pins in `outputPins`, not
every wired board pin. Leaves INPUT pins (analog sensors on A0,
pull-down dividers, etc.) free for the SPICE solver instead of being
shorted to 0 V by an ideal MCU V-source.
- start.ts: extended __spiceDebug to also expose outputPinsByBoard +
nodeVoltages + pinNetMapEntries for the visual harness.
- ESP32 / RP2040 / RISC-V / C3 simulators: pass `'mcu'` source flag to
triggerPinChange / setPinState so the new outputPins tracking fires
on those boards too (was AVR-only before).
- useSimulatorStore: stopBoard/resetBoard call pm.resetPinStates() so
outputPins clears between runs; Esp32Bridge.onPinChange passes the
`'mcu'` flag in all three places it's wired.
- types/board.ts: ATtiny85 FQBN `clock=internal16mhz` →
`clock=16pll` (ATTinyCore 1.5.2 renamed the option).
Backend
- esp-idf-template/main/CMakeLists.txt: skip the
`-DLED_BUILTIN=2` fallback for esp32c3 and esp32s3 targets. Both
variants already define LED_BUILTIN in pins_arduino.h via a
self-define macro (`#define LED_BUILTIN LED_BUILTIN` + `static const
uint8_t LED_BUILTIN = ...;`). Pre-defining the symbol from the
command line expanded the static-const declaration to
`static const uint8_t 2 = ...;` — a syntax error that broke every
ESP32-C3 / S3 build (`expected unqualified-id before numeric
constant`).
Examples
- examples.ts: bulk-fix 72 wire endpoints that referenced
`componentId: 'nano-rp2040'` / `'esp32-c3'` etc. (boards that don't
exist on the canvas). Replaced with `'arduino-uno'` (the canvas
board-id convention) and converted `D<n>` pin names to `GP<n>` for
Pico-style boards. Affects pico-blink, pico-i2c-scanner,
pico-i2c-rtc-read, pico-spi-loopback, c3-blink and others.
Tests
- scripts/visual-led-test.mjs: CDP-driven harness. Default suite covers
Blink (single-pin), Button (idle-OFF invariant — catches the
INPUT_PULLUP regression), Traffic-Light (multi-pin sequencing),
Fade-LED (PWM gradient — ≥3 distinct levels), RGB-LED (≥3 PWM pins
driven). Run via `npm --prefix frontend run test:visual` against a
Chrome on `:9222` + vite on `:5174` + backend on `:8001`.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Backend:
- api/routes/compile.py accepts board-specific compile options
and dedups in-flight identical requests
- services/espidf_compiler.py expanded ESP-IDF wrapper with the new
options surface (sdkconfig.defaults.in
template added)
- services/arduino_cli.py honour the new options envelope
- services/esp32_lib_bridge.py thread board options through to QEMU
Tests:
- tests/test_compile_request_dedup.py end-to-end dedup behaviour
- tests/test_espidf_options.py covers the new options parsing
Frontend:
- services/compilation.ts client-side mirror — sends the new
options field on every compile request
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the deferred Phase 3.3. Root-causes the Pi 2 "Attempted to
kill init" panic as `mount /dev/vda` failing with EINVAL — Debian
armmp does not have ext4 builtin (only fuseblk in /proc/filesystems).
- qemu_manager: PI_CONFIGS gains raspberry-pi-zero / -1 / -2 entries.
All three use the armmp armhf kernel + Cortex-A7 CPU + the mmio
virtio transport (arm-32 virt PCI fails -75 due to missing reg DT
property). Pi Zero / Pi 1 get the small 1-core / 512 MB profile;
Pi 2 gets 4-core / 1 GB. QEMU command builder branches on cfg.bus
for virtio-blk-pci vs virtio-blk-device (and serial likewise).
- manifest.json: new `raspberry-pi-armhf` image_set wiring three
assets (kernel + initramfs + zstd rootfs).
- Frontend BoardKind gains the three new kinds + an isPiBoardKind()
helper. Replaces the eight scattered `=== 'raspberry-pi-3' ||
=== 'raspberry-pi-4' || === 'raspberry-pi-5'` branches in
useSimulatorStore, Interconnect, loadExample, boardProtocols.
ComponentRegistry gets three new picker entries.
- board-kinds-coverage test: ACCEPTED_UNCOVERED gains the new kinds
(backend boards have no canvas examples).
The matching armhf build-pi-kernel.sh / build-pi-rootfs.sh changes
live in velxio-prod's scripts/ (private overlay) — the upstream
kernel build script only knows about arm64; armhf is built in the
private repo because the assets ship through the license endpoint.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the public extension points the velxio-prod overlay uses to bind
real canvas-side I2C/SPI/UART models (BME280, future MCP23017, etc.)
to a running Pi guest's protocol shims:
- qemu_manager: set_pi_slave_handler(fn) / get_pi_slave_handler() for
pi_attach_slave + pi_detach_slave WebSocket messages. OSS image
leaves the hook unset so the messages are silently dropped.
- simulation route: parses the two new WS message types and forwards
them to the registered handler when present.
- RaspberryPi3Bridge: attachSlave(spec) / detachSlave(spec) frontend
side of the protocol.
- piSlaveScanner: at simulation start walks components + wires,
identifies I2C/SPI/UART peers wired to Pi protocol pins (40-pin
header physical-pin numbering), and emits one attach per
bus/address pair (deduped across SDA+SCL wires).
- RaspberryPiWorkspace: invokes the scanner once the bridge is open,
with retries to ride out the WS-still-connecting race.
- integration test: pi3_bme280_attach.py boots the Pi, pre-attaches a
BME280 via the slave handler, runs a host-side proto loop, runs
guest python smbus2.read_byte_data(0x76, 0xD0) and asserts the
console reads back CHIP=0x60.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Backend: extract per-board config into a PI_CONFIGS dict keyed by
board_type. Pi 3/4/5 share the same arm64 image set (kernel +
initramfs + rootfs) and differ only in QEMU -cpu and -m:
raspberry-pi-3 → cortex-a53 + 1G (BCM2837, ARMv8 64-bit)
raspberry-pi-4 → cortex-a72 + 2G (BCM2711, ARMv8 64-bit)
raspberry-pi-5 → cortex-a76 + 2G (BCM2712, ARMv8 64-bit)
PiInstance now carries board_type so the per-board lookup happens
once at start_instance time. Unknown board_type falls back to
DEFAULT_PI_BOARD ('raspberry-pi-3') instead of erroring out (for
back-compat with older clients).
Pre-warm hook walks every unique image_set in PI_CONFIGS so the
provider only downloads each set once even when several Pi models
are registered.
Frontend:
- BoardKind union gains 'raspberry-pi-4' and 'raspberry-pi-5'.
- BOARD_KIND_LABELS + BOARD_KIND_FQBN entries for both new boards
(FQBN null since they use the Pi VFS + Python toolchain like Pi 3).
- ComponentRegistry inserts two new component metadata entries
cloning the Pi 3 board art with different thumbnail colours.
Tag name reused so the same velxio-raspberry-pi-3 web element
draws the board on the canvas — the 40-pin GPIO layout is
identical across Pi 3/4/5.
- boardProtocols.ts: Pi 3/4/5 share the BCM physical→GPIO table
(PI3_BCM) since the 40-pin header layout is identical.
- loadExample.ts: where 'raspberry-pi-3' is special-cased (VFS
ingest, .cpp vs .ino filename), now matches Pi 3/4/5 alike.
- Interconnect.isPi3Bridge() recognises all three Pi family members
so Arduino↔Pi serial routing keeps working.
- RaspberryPi3Bridge constructor gained a boardKind parameter
defaulting to 'raspberry-pi-3'. The WebSocket 'start_pi' message
now ships the actual board kind so the backend knows which
PI_CONFIGS entry to use.
- useSimulatorStore.addBoard wires bridge construction for all
three Pi family members.
Pi Zero/Pi 1/Pi 2 (armhf) come in Phase 3.3 — separate kernel
package + armhf rootfs build, no change here.
Smoke-tested inside the prod container:
Pi 4 (cortex-a72) → reached agetty login on hvc0
Pi 5 (cortex-a76) → reached agetty login on hvc0
Both show 'aarch64' in uname -m.
The Phase 2 E2E test was sending the Python GPIO command via
'python3 -c "..."' but bash quote-nesting silently corrupted the
script — the python process started, printed nothing, exited 0, and
the test asserted 'GPIO_SETUP 17 out' was missing in proto bytes
(it never got sent because the python script never ran).
Switch the test to base64-encode the script + pipe through base64 -d
into a file, then execute. Verified end-to-end now:
[test] proto received 36 bytes:
GPIO_SETUP 17 out pud_off
GPIO 17 1
[test] ✓ shim → proto pipeline works
Also bump the rootfs manifest entry to the final Phase 2 build
(d6d4a274 raw / debd1c33 zst, version 2026.05+phase2-shims-final).
Earlier auto-discovery in _transport.py was hanging at import time
on some glob/sysfs interaction. Now hardcoded /dev/vport1p1 which
is the empirical path under -M virt + virtio-blk-pci on slot 0.
QEMU 10's virtserialport on a socket chardev (server=on,wait=off)
silently drops guest→host bytes. Reproduced cleanly: writes from
inside the guest to /dev/vport<N>p<M> succeed (no errno) but the
connected client socket receives 0 bytes. Same bug whether the
client is a single recv loop, multiple threads, TCP or UNIX socket,
or whether QEMU runs as server vs client. virtconsole on the same
socket works fine — only virtserialport is broken.
Workaround: use `pipe` chardev (a pair of named FIFOs created
beforehand by qemu_manager). guest→host through .out flows reliably
in QEMU 10 — verified with manual test: 'echo PIPE_TEST > /dev/vport1p1'
in the guest produces 'PIPE_TEST\n' immediately on the host side.
Changes:
- qemu_manager._boot: allocate a temp basename, mkfifo .in + .out,
pass to QEMU as 'pipe,path=<base>'.
- qemu_manager._connect_gpio: open both FIFOs O_RDWR | O_NONBLOCK on
host side (O_RDWR keeps the FIFOs open even when guest hasn't
opened its side yet), wire .out into asyncio via loop.add_reader.
- qemu_manager._reply_gpio / _send_gpio: write to .in fd via os.write.
- qemu_manager._handle_gpio_line: extended Phase 1 GPIO-only parser
into a full Phase 2 mux: GPIO/GPIO_SETUP/GPIO_IN/PWM_*/I2C/SPI/UART
with appropriate replies.
- qemu_manager._shutdown: close FDs + unlink the FIFOs.
- manifest.json: bump raspberry-pi-3-virt rootfs to 2026.05+phase2-shims
(the new rootfs ships the velxio shim Python modules under
/usr/lib/velxio-shims/).
raspi3b pl011 RX is broken in QEMU 10 + kernel 6.12 — see
project/pi-emulation/decisions.md for the full debugging trail.
This commit lands Phase 1 of the rebuild: switch the QEMU machine
to virt + cortex-a53, boot the velxio kernel/initramfs/rootfs over
virtio-blk-pci, and expose the user shell on /dev/hvc0 via
virtio-serial-pci + virtconsole.
End-to-end smoke verified: boot → agetty autologin → bash prompt →
echo round-trip returns the typed token. Tested inside the prod
container with QEMU 10.0.8 and our cloud-derived kernel 6.12.88.
What changed:
backend/app/services/qemu_manager.py
PI3_IMAGE_SET -> raspberry-pi-3-virt
PI3_KERNEL_NAME / PI3_INITRAMFS_NAME / PI3_ROOTFS_NAME new
QEMU cmd rewritten end-to-end:
-M virt -cpu cortex-a53 -smp 4 -m 1G
-kernel <velxio-kernel-arm64> -initrd <velxio-initramfs-arm64.cpio.gz>
-drive ... -device virtio-blk-pci (NOT virtio-blk-device — mmio
variant left /dev/vda unregistered)
-nic none -display none -monitor none -serial none
-chardev socket... -device virtio-serial-pci -device virtconsole
(user console -> /dev/hvc0)
-chardev socket... -device virtserialport,name=velxio-protocol
(Phase 2 channel -> /dev/vport0p2)
No -dtb (virt generates its own), no -append init=... (kernel runs
our initramfs which then switch_root to rootfs and exec's its
/sbin/init — Alpine OpenRC).
backend/app/services/boot_images/manifest.json
New image set raspberry-pi-3-virt with three assets uploaded via
the existing license-endpoint pipeline. Old raspberry-pi-3 entry
flagged deprecated:true and kept for one release for rollback.
test/pi3_console_boot/test_pi3_console_boot.py
Updated QEMU argv to match qemu_manager exactly. Markers now look
for the Velxio Pi Simulator MOTD + 'login on hvc0' (autologin
proof). Round-trip echo still required to pass.
PiTerminal didn't call term.focus() on mount, so xterm.js stayed
passive — onData only fires when the DOM element has focus. Users
saw the boot prompt but their keystrokes went to whatever element
held focus when they clicked Run (canvas, code editor), never
reaching the bridge. Calling focus() right after fit() makes the
prompt receive input the moment it's visible.
The qemu_manager change adds INFO-level logging when serial_input
WebSocket messages reach send_serial_bytes — useful diagnostic for
future Pi3 input problems (proves whether bytes reached the backend
before we look at TTY / kernel / PL011 wiring).
User report: on the solar-tracker project (5218f9e3) only one servo
moved and the log showed `ch=0 duty=X% gpio=12` (wrong — servoPan was
attached to GPIO 13) and `ch=1 ... gpio=-1` (servoTilt's channel
never resolved).
Root cause traced through the GPIO Matrix dump: the firmware does
exactly what the Arduino-ESP32 Servo library says — `ledcAttachPin(
13, 0)` writes signal 71 (LEDC_HS_SIG_OUT0) into `gpio_out_sel[13]`,
and `ledcAttachPin(12, 1)` writes signal 72 (LEDC_HS_SIG_OUT1) into
`gpio_out_sel[12]`. Per the ESP32 Technical Reference Manual section
4.11, Table 4-3:
71 .. 78 → LEDC HS channels 0..7
79 .. 86 → LEDC LS channels 0..7
The legacy worker code at esp32_worker.py:426 used the off-by-one
range `72 <= signal <= 87` with `ledc_ch = signal - 72`. The mistake
masked itself for single-servo projects because the 0x5000 duty
callback's channel index was internally consistent with the bogus
math, so the duty STILL reached the correctly-routed pin (just
labelled wrong). The new SignalRouter unit tests caught the
discrepancy the moment two servos drove distinct channels: signal
71 (HS_CH0, gpio 13) was REJECTED by the off-by-one filter and
signal 72 (HS_CH1, gpio 12) was misclassified as channel 0.
When I ported the legacy range into `esp32_signals.SIG_LEDC_HS_CH0_OUT_IDX`
the bug came along for the ride. Fix both modules:
* `backend/app/services/esp32_signals.py`: HS 71-78, LS 79-86.
* `frontend/src/simulation/esp32-signals.ts`: mirror.
* tests updated; 20 backend + 23 frontend pass.
After deploy the user's two servos will resolve to their declared
pins:
ch=0 duty=X% gpio=13 (servoPan, was wrongly emitting gpio=12)
ch=1 duty=X% gpio=12 (servoTilt, was wrongly emitting gpio=-1)
This is also why the multi-servo blink "patch" in commit 77bf897
appeared to help: with both pins ALIASED to the same channel via
the off-by-one, the broadcast fallback was the only thing producing
ANY movement on the second servo at all.
The esp32_worker.py subprocess is launched via `python <abs_path>`
and runs with a sys.path that does NOT include the backend/ package
root, so `from app.services.signal_router import SignalRouter`
raised ModuleNotFoundError at worker startup. The worker exited
with code 1 before QEMU even loaded, and the frontend surfaced the
generic "ESP32 crash detected — cache error" banner.
Mirror the existing esp32_flash_image fallback pattern (already in
this same file): try the package import first, fall back to
importlib.spec_from_file_location with the sibling .py path, then
publish the resulting module under its bare name in sys.modules so
typing references continue to work.
Verified: a synthetic test that strips backend/ from sys.path can
still construct a SignalRouter via the fallback. 20 unit tests in
test_signal_router.py still pass.
Replaces the per-peripheral ad-hoc `_ledc_gpio_map` cache with a
proper signal-routing abstraction that mirrors the ESP32 SoC's
IO_MUX + GPIO Matrix exactly. Same idea as real silicon: signal
sources (LEDC channels, RMT, MCPWM, ...) → 40-entry routing table
→ GPIO pins.
Motivation (from user bug report in
velxio.dev/project/5218f9e3-136d-43b3-bba1-6cebde21e1a4): two
ESP32 servos on a solar-tracker visibly oscillated between two
positions instead of moving smoothly when the user changed LDR
sliders. Commit 77bf897 patched it (per-channel gpio memo +
broadcast guard) but the user requested a proper hardware-fidel
architecture, not patches.
Backend:
* `app/services/signal_router.py` — SignalRouter class. Forward
index (gpio → signal_id) + reverse index (signal_id → set of
gpios). `replace_snapshot()` returns the diff for the polling-
fallback path; future C plugin hook becomes a push without
touching this code.
* `app/services/esp32_signals.py` — Signal id constants from
ESP32 TRM (LEDC HS 72-79, LS 80-87) + `ledc_signal_for_channel()`
helper.
* `app/services/esp32_worker.py` — `_ledc_gpio_map` is gone;
`_refresh_ledc_gpio_map` replaced by `_refresh_signal_routing`
which emits `gpio_routing {gpio, signal_id}` events on diff.
The 0x5000 LEDC callback and the LEDC poll thread now emit
`ledc_duty {channel, duty_pct}` (canonical, no gpio) alongside
the legacy `ledc_update {channel, duty, gpio}` for back-compat
during rollout.
Frontend:
* `simulation/SignalRouter.ts` — 1-to-1 TS mirror of the Python
class. Same forward + reverse index; same `pinsForSignal` /
`updateRouting` / `clearRouting` API.
* `simulation/esp32-signals.ts` — Signal id constants, mirror
of the Python module.
* `simulation/Esp32Bridge.ts` — new `onLedcDuty`, `onGpioRouting`,
`onGpioRoutingClear` callbacks; handlers for the new event types.
* `store/useSimulatorStore.ts` — `makeLedcDutyHandler` looks up
pins via `router.pinsForSignal(ledcSignalForChannel(channel))`
and dispatches per pin. `makeGpioRoutingHandler` /
`makeGpioRoutingClearHandler` keep the mirror in sync. Per-board
`signalRouterMap` parallels `pinManagerMap` in lifecycle.
`makeLedcUpdateHandler` (and its memo workaround from 77bf897)
stays wired for back-compat during rollout; removed in a
follow-up commit once prod is verified stable on the new path.
Tests:
* `test/backend/unit/test_signal_router.py` (20 tests) covers
update/clear semantics, idempotency, multi-pin routing,
snapshot diff, channel↔signal-id helpers, and the multi-servo
regression scenario.
* `frontend/src/__tests__/SignalRouter.test.ts` (17 tests) is the
mirror — same scenarios on the TS side.
* `frontend/src/__tests__/esp32-multi-servo-gpio-matrix.test.ts`
(6 tests) drives the end-to-end SignalRouter handler pipeline,
asserts that two servos on GPIO 13/12 via LEDC channels 0/1
move independently (no mirroring), that re-routing carries
cleanly, and — critically — that `PinManager.broadcastPwm` is
never called.
Totals: +700 LOC, 1876 frontend tests pass (was 1853), 278 backend
unit tests pass (was 259).
Docs: ESP32_EMULATION.md §9.2 rewritten with the new architecture
diagram + a runbook for adding future peripherals through the
SignalRouter.
The C plugin hook in qemu-lcgamboa that would push gpio_out_sel
writes synchronously (eliminating the polling race window entirely)
is the next step — kept as a follow-up because the polling-fallback
path here already resolves the routing before each duty event
fires, so the bug is fixed end-to-end. The plugin work removes the
race condition fundamentally.
User-reported bug: Pi 3 simulator showed boot output but keyboard
input was ignored — the shell was effectively read-only.
Root cause: velxio-init's bash redirect was `</dev/console
>/dev/console`. From userspace, /dev/console is write-only — it is
the kernel's printk target and accepts writes (so we saw boot output
fine) but reads return EOF / block forever. Bash never saw a
keystroke and the user couldn't type.
Fix: read AND write through /dev/ttyAMA1. The 12 s devtmpfs-wait
already in velxio-init guarantees the device node exists by the
time the shell-respawn loop runs. setsid -c still gives bash a
controlling terminal so PS1, job control, and Ctrl-C all work.
Manifest version bumped to 2026-04-21+ttyAMA1; sidecar SHA check
invalidates the cached SD on every velxio backend so the fix lands
without an operator dance.
The previous velxio-init was racing devtmpfs population: its bash
redirect '</dev/ttyAMA1 >/dev/ttyAMA1' fired before the kernel had
enumerated the PL011 driver and populated the device node, so PID 1's
fd 0/1/2 redirect failed and the `while true` loop spun on
"No such file or directory" forever.
Two fixes baked into the SD image:
1. Wait up to 12 s for /dev/ttyAMA1 to appear (200 ms poll × 60).
On real bare-metal Pi the node is there at init time, but
under QEMU emulation the PL011 probe races.
2. Exec the shell with </dev/console >/dev/console — /dev/console is
set up by the kernel (no race) and points at the last `console=`
arg from the cmdline, which is ttyAMA1. Also wrap in `setsid -c`
so bash gets a controlling terminal and behaves interactively.
Verified end-to-end with a live QEMU boot against the patched .img:
shell prompt `root@raspberrypi:/#` appears within ~50 s wall (most
of that is the kernel waiting on the second SD slot mmc1 timeout
twice = 20 s).
Pi 3 simulator boot through Pi OS systemd graph was unworkable inside
QEMU's raspi3b emulation:
* The PL011 UART at 0x3f201000 enumerates as ttyAMA1 (not ttyAMA0 —
the mini-UART at 0x3f215040 takes ttyAMA0 and fails to probe under
QEMU). After ~9 s of kernel time the boot effectively went silent
on the serial: earlycon was disabled by the normal console init
and the IRQ-driven serial driver loses TX under QEMU's emulation.
* Even with `keep_bootcon`, systemd dependency graph took 2-3 min to
walk inside emulation (network waits, tmpfiles, journald,
hostname/machine-id randomness). Masking 9 boot-blocking units
helped but didn't fix the silent-after-9s problem.
Solution: skip systemd. The SD image is now baked with
`/usr/local/sbin/velxio-init` (a 30-line bash script) and the kernel
cmdline points init= at it. velxio-init mounts /proc /sys /dev /pts
/run /tmp, sets hostname, then loops a passwordless `/bin/bash
--login </dev/ttyAMA1 >/dev/ttyAMA1`. User sees the prompt within
~10 s of clicking Run; Ctrl-D respawns a fresh session.
Cmdline additions:
- `keep_bootcon` — keep earlycon alive after the regular console
registers, so kernel printk continues to reach ttyAMA1.
- `console=ttyAMA1,115200` — the correct PL011, not ttyAMA0.
- `init=/usr/local/sbin/velxio-init` — bypass systemd entirely.
Python, GPIO shim, apt, mount, etc. all work — they don't need
systemd as PID 1, just a populated rootfs + mounted pseudo-fs.
Manifest version bumped to 2026-04-21+velxio-init. Same byte size,
different SHA, so the sidecar-based cache invalidator forces a
re-fetch on every velxio backend the next time it starts.
Boot from cold to root prompt was 2-3 min because Pi OS Trixie waits
on a handful of services that timeout instead of completing:
- systemd-networkd-wait-online (60s default)
- NetworkManager-wait-online (30s default)
- wpa_supplicant + dhcpcd5 (no usable interfaces)
- raspi-config / firstboot / userconfig (no point in QEMU)
The SD image was re-baked through scripts/configure-pi3-autologin.sh
with all of them masked (the script grew a `mask_unit` helper that
symlinks each unit to /dev/null inside the rootfs). Login prompt now
appears in ~30s wall.
New manifest version 2026-04-21+autologin+fastboot — same byte count
as the previous build (still 5.4 GiB raw) but a different SHA so the
sidecar-based cache invalidation forces every container to refetch.
Two more defects making Pi 3 boot silently:
1. The kernel8.img that ships in the Pi OS armhf boot partition is a
gzip-compressed PE-COFF Image (first 4 bytes 0x1f8b0800). QEMU's
`-kernel` does NOT auto-decompress; it tries to execute the gzip
header as ARM code and the CPU faults immediately. Result: zero
bytes on ttyAMA0, simulator looks dead. Switch the asset_id to a
pre-decompressed kernel (24 MiB raw vs 9.7 MiB gzipped) so QEMU
gets a valid Image to boot.
2. Even with a real kernel, the original cmdline `console=ttyAMA0`
alone wasn't enough — the kernel can't initialise the BCM2837
PL011 UART early enough for `printk` to reach the serial console
under QEMU's bare-metal boot (no Pi firmware to set it up
beforehand). Adding `earlycon=pl011,mmio32,0x3f201000` makes the
kernel program the UART itself in the early boot path.
Verified: boot output starts streaming within 100 ms of QEMU
launch instead of never.
The cmdline also locks the baud rate at 115200 to match the agetty
drop-in created by scripts/configure-pi3-autologin.sh.
User report: clicked Pi 3 board → nothing visible happens. Three
defects, all on the same path:
1. The kernel cmdline carried over from the original pre-OSS-split
code: `quiet init=/bin/sh`. Result: kernel boot messages
suppressed, then dropped straight to bare /bin/sh with no PS1 so
the user sees an empty serial. Removed both. The kernel cmdline
is now just `console=ttyAMA0 root=/dev/mmcblk0p2 rootwait rw
dwc_otg.lpm_enable=0`, which lets systemd start a real
serial-getty@ttyAMA0.service.
2. Pi OS Trixie armhf since Bookworm ships without a default user
(no more pi/raspberry). With cmdline #1 fixed, the user would
land at a login prompt and be stuck. Fix: pre-bake a systemd
drop-in at /etc/systemd/system/serial-getty@ttyAMA0.service.d/
autologin.conf that uses `agetty --autologin root` so the serial
console drops to a root shell on first prompt. The browser
canvas IS the authentication boundary; the SD image is mounted
RO via a qcow2 overlay so per-session edits don't persist.
Edit happens in velxio-prod/scripts/configure-pi3-autologin.sh
(to follow in a separate commit).
3. Architectural: the original cache-hit probe was size-only.
Today's SD image rebake produced a file with identical byte count
but different SHA256 — the cache served stale content for every
request even after a manifest bump. Fix: write a sidecar
`<file>.sha256` after every successful materialise and trust it
on subsequent probes. Manifest SHA bumps invalidate the cache
regardless of size. Two regression tests guard this:
- test_provider_sidecar_invalidates_on_sha_mismatch
- test_provider_missing_sidecar_treats_file_as_invalid
Manifest bumped to version "2026-04-21+autologin" for the SD image
(kernel + DTB unchanged, still 2026-04-21).
Pi 3 simulation had been broken since at least April 2026 (51
fail-events / 24h per docs/PI3_EMULATION_BROKEN.md). Two distinct
defects compounded:
1. qemu_manager.py hard-coded paths for kernel8.img, a device-tree
blob (under a DOS 8.3 short name!), and a 5.4 GiB Raspberry Pi OS
SD image — none of which shipped in the repo or were pulled at
image build.
2. qemu-system-arm + qemu-utils were missing from the Docker image
entirely, so even with the boot files in place QEMU couldn't
launch. Add both to Dockerfile.standalone (~200 MB).
The architecture fix is a new `app.services.boot_images` module:
* Manifest-driven (boot_images/manifest.json, versioned in repo,
declares SHA256 + size for each file, supports an optional
`compressed.{encoding,sha256,size_bytes}` block for assets shipped
as .zst).
* `BootImageProvider` materialises files lazily, atomically (temp +
rename), verifies SHA256 pre- AND post-decompression, caches under
/var/cache/velxio/boot-images, serialises concurrent get() calls
per image set via asyncio.Lock.
* `AssetDownloader` Protocol with two impls:
- `LicenseGatedDownloader` — same flow ESP32 / RISC-V QEMU libs
use (VELXIO_BINARY_BASE_URL + VELXIO_LICENSE_KEY).
- `LocalDirectoryDownloader` — for tests + in-prod use where the
licence-module storage is already on the same filesystem (saves
the loopback HTTP roundtrip on a 1.4 GiB blob).
* `build_downloader_from_env()` picks one — local-dir wins if both
sets of env vars are present, so the prod box short-circuits to
direct disk reads automatically.
* Lifespan hook in qemu_manager.py pre-warms the cache on container
boot so first-time user requests don't pay the 30-60 s download
+ decompress latency.
Adding a future board kind (Pi 4 / Pi 5) is now: upload assets via
upload-binary.sh, append an entry to manifest.json, register a
lifespan pre-warm in the new board's service module. Zero edits to
provider.py / downloader.py.
Manifest entries for raspberry-pi-3:
kernel8.img 9 695 883 bytes (uncompressed)
bcm2710-rpi-3-b.dtb 34 687 bytes (uncompressed)
raspios-trixie-armhf.img 5 729 419 264 bytes raw
/ 1 488 002 803 bytes .zst on wire (zstd -19)
source: 2026-04-21 build from raspberrypi.com
Tests: 21 new unit tests covering manifest parsing, integrity
helpers, both downloaders, and the provider's idempotent /
concurrent / integrity / decompression / warmup paths. In-process
FakeDownloader keeps the suite under 1 s and httpx-free.
Docs: new docs/BOOT_IMAGES.md describes the architecture, on-disk
layout, named-volume operation, and the procedure for adding a new
image set.
After Phase 4 of the OSS / pro split, the OSS code base imports zero
auth/DB modules (verified with grep across backend/app/). But the
requirements.txt + config.py + .env.example + docs still listed
SQLAlchemy, aiosqlite, JWT/bcrypt, OAuth, SECRET_KEY etc. as if they
were live. Self-hosters running `pip install -r requirements.txt`
were pulling ~30 MB of packages the code never imports.
Changes:
* backend/requirements.txt — drop sqlalchemy, greenlet, aiosqlite,
python-jose, passlib[bcrypt], bcrypt, authlib, email-validator,
python-multipart. Keep fastapi, uvicorn, websockets, pydantic,
pydantic-settings, httpx, mcp, esptool, wasmtime — everything OSS
actually uses.
* backend/app/core/config.py — Settings reduced to FRONTEND_URL only.
Comment explains the overlay path that adds the rest at Docker
build time.
* backend/.env.example — same trim: only FRONTEND_URL, with a comment
explaining why this file is almost empty.
* README.md — "Auth & Project Persistence" section rewritten to
describe .vlx export/import. Env-var table reduced to a single row.
Stack table updated: no SQLAlchemy, no JWT, persistence = .vlx
files.
* CLAUDE.md — intro line updated (Auth: None, persistence: .vlx).
Key-file-locations rewritten to list the OSS-stateless backend +
the new lib/proRoutes / proSession / proSaveAction seams, with an
explicit "removed in the split" note pointing to velxio-prod.
Stores section drops useAuthStore (overlay-only now). Backend
gotchas drop the bcrypt + email-validator + model-import notes.
Implemented-features list replaces "Auth + URL persistence + user
profile" with portable .vlx export/import.
* docs/ESP32_EMULATION.md — two `docker run` examples dropped the
`-e SECRET_KEY=...` arg (no longer needed).
OSS build verified end-to-end (285 SEO pages prerender, 20 stateless
routes, zero sqlalchemy imports).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Without the type annotation, FastAPI treats `request` as a Query
parameter and bubbles it up to every endpoint that uses
`Depends(get_current_user_id)`. Result: POST /api/compile/start
and POST /api/compile/ both returned 422
{"loc":["query","request"],"msg":"Field required"} on every call
the frontend made — compile was fully broken in production.
The frontend then caught the 422 axios error and surfaced
response.data as a CompileResult, which had no success/stdout/
stderr/error fields, so the editor's CompilationConsole rendered
only the fallback "✕ Compilation failed" line with no detail.
Annotating `request: Request` is the standard FastAPI pattern;
the framework injects the raw HTTPRequest and no longer treats
it as a query parameter.
Phase 2 of the OSS / pro split. The hook seams introduced in Phase 1
let stateless routes (compile, libraries, simulation, iot_gateway)
run without the auth/DB stack importable. Now we actually delete the
stack:
app/api/routes/auth.py
app/api/routes/projects.py
app/api/routes/admin.py
app/api/routes/metrics.py
app/models/{user,project,usage_event,password_reset_token}.py
app/schemas/{auth,admin,project}.py
app/core/{dependencies,security}.py
app/database/session.py
app/services/{metrics,odoo_mail,project_files}.py
app/utils/{geo,slug,boards}.py
Private deployments (velxio.dev) get the same modules back via the
velxio-prod overlay: pro/backend/app/api/routes/auth.py etc. are
COPYed onto /app/... at container build time, and register_pro()
includes their routers + registers the lifespan/metrics/auth hooks.
main.py shrank back to the stateless router includes + a single
`run_lifespan_startup()` call. The Phase-1 try-import block that wired
record_compile / get_current_user_id from upstream is gone — those
adapters live in pro now.
Verification:
OSS only: 20 routes (compile, libraries, simulation, gateway).
OSS + pro: 94 routes — identical to pre-refactor velxio.dev.
Net change: -2400 lines from OSS, all of which moved to velxio-prod's
overlay. Self-hosted OSS users lose accounts + project persistence;
the Phase 4 .vlx export/import gives them a portable replacement.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First phase of the OSS / pro split. Goal: open the seams so the auth/DB/admin
stack can move into the private overlay (Phase 2-3) without the routes that
stay in OSS (compile, libraries, simulation, iot_gateway) having to know.
Backend
-------
* New app/core/hooks.py — registry for record_compile, get_current_user_id,
and lifespan startup tasks. Each hook is a no-op by default; overlays
call register_* in register_pro(app) to plug in a real implementation.
* compile.py now imports only from app.core.hooks. Drops the direct deps on
app.core.dependencies, app.database.session, app.models.user, and
app.services.metrics. Route signatures use `Depends(get_current_user_id)`
instead of `Depends(get_current_user)`; the metric helper passes user_id
through rather than a User instance.
* compile_chip.py drops the unused _current_user Depends entirely.
* main.py wraps the auth/DB stack import in try/except. When it succeeds
(today's behavior on velxio.dev), an adapter bridges record_compile and
get_current_user_id to the existing app.services.metrics + dependencies,
and the create_all + ALTER TABLE migration block runs via a registered
lifespan_startup hook. When it fails (the post-Phase-2 OSS image), main
logs "running stateless" and skips registering anything — the routes
still load and behave as no-ops for metrics + always-anonymous for auth.
Frontend
--------
* useAutoSaveProject becomes a skeleton: one useState + one useEffect that
delegates to an installed AutoSaveImpl. installAutoSaveImpl() replaces
the impl without changing hook count, so React's rules-of-hooks stay
satisfied even after the impl moves out of OSS.
* New hooks/autoSaveImpl.ts holds the original logic (debouncing, dirty
detection, owner eligibility, fetch keepalive on unload), refactored to
emit() instead of useState. It self-registers at module load; main.tsx
imports it for the side effect.
* AppHeader wraps the entire user-vs-login UI in a data-velxio-slot
="header-auth" boundary. Today the OSS UI still renders inside the slot
— the overlay can portal-inject additional items now, and in Phase 3
the slot becomes the sole owner of header auth UX.
Behavior is identical on velxio.dev (pro overlay imports everything
successfully, every adapter wires up). The change is purely structural:
deleting the auth/DB modules tomorrow no longer crashes OSS at import.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
arduino-esp32 v2.0.17's libraries/WiFiClientSecure/src/ssl_client.cpp:23
wraps its entire body in:
#if !defined(MBEDTLS_KEY_EXCHANGE__SOME__PSK_ENABLED) \
&& !defined(MBEDTLS_KEY_EXCHANGE_SOME_PSK_ENABLED)
# warning "Please call idf.py menuconfig ..."
#else
ssl_init / start_ssl_client / stop_ssl_socket /
send_ssl_data / get_ssl_receive / data_to_read
#endif
Our esp-idf-template/sdkconfig.defaults did not enable any PSK key-exchange
mode, so MBEDTLS_KEY_EXCHANGE_SOME_PSK_ENABLED was never auto-set by
mbedtls and ssl_client.cpp compiled to an empty translation unit. The
companion WiFiClientSecure.cpp still compiled and ended up in
libarduino-esp32.a with dangling references, breaking the link of every
sketch that pulls in HTTPClient or WiFiClientSecure (directly or
transitively).
Reproduced against the user's WiFi + HTTPClient example.com sketch on the
prod server and again locally with ESP-IDF v4.4.7 + arduino-esp32 v2.0.17;
the prebuilt sdkconfig that ships with arduino-esp32 itself sets both
flags, so we just align with that.
After the fix the same sketch links cleanly:
velxio-sketch.bin binary size 0xbf470 bytes ... 25% free
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the register-then-immediately-forgot race for good. Two parallel
calls to /velxio/api/send-welcome and /velxio/api/send-password-reset
were hitting Odoo's REPEATABLE-READ snapshot timing: both workers took
their snapshot BEFORE either had committed the partner row, so each
ran its own INSERT and the second blew up on the velxio_user_id
unique constraint — costing one of the two emails.
Add a new sync_partner() helper that POSTs to a new Odoo route
/velxio/api/upsert-partner (lives in velxio_subscription) which only
upserts the partner — no mail, no subscription, fast. Register and
forgot-password await this BEFORE firing the async welcome / reset
tasks. Each user's flow is sequential at the Velxio HTTP layer, so
the snapshot race disappears.
sync_partner() reuses the existing _post() error-swallowing pattern.
If Odoo is down, sync_partner returns None and the await is a no-op
— the user still registers / gets the generic 200, the welcome /
reset endpoints retain their defensive upsert as a fallback path.
Adds velxio_user_id to the send_password_reset payload (mirroring
send_welcome). The Odoo side's res_partner.velxio_user_id is unique,
so when Odoo eventually upserts on this endpoint the constraint
serializes concurrent register-then-immediately-forgot upserts and
prevents the duplicate-partner record the previous wire format risked.
Closes the remaining gaps in cross-board I2C so any topology of
supported boards (Uno↔ESP32, two ESP32s, Uno↔Uno↔Uno, ESP32-C3
connected to anything, etc.) works end-to-end with all I2C
components including write-only sinks (SSD1306, PCF8574, LCD-I2C).
Implementation (6 phases):
1. **BFS routing in I2CBusManager**: connectToSlave + handleExternalConnect
walk the bridge graph with a visited Set so multi-hop chains
(A↔B↔C with the device on C) resolve transparently. A new
forwarder-device shim is installed at intermediate hops so the
existing handleExternalWrite/Read/Stop machinery routes
through without per-method visited tracking.
2. **Per-peer proxy ownership in Esp32BridgeShim**: replaces the
global _proxiedAddrs Set with _proxiedByPeer Map so concurrent
bridges to the same ESP32 (e.g. wired to both Uno and Pico)
don't wipe each other's proxies on teardown. Interconnect's
per-wire teardown calls clearProxiesForPeer(peerBus) instead of
clearAllProxies.
3. **BFS-aware proxy sync**: syncProxyFromPeer now walks the peer
bus + its transitive bridges, so an ESP32 sees devices on
boards two or more hops away. _peerDeviceLookup keeps a flat
addr → device map for write-forwarding and resync.
4. **Periodic resync (250 ms)**: Esp32BridgeShim runs a setInterval
while any proxy is live, re-dumping each device with
dumpRegisters() and pushing updateProxyI2c only when an XOR-
stride hash changes. This keeps RTC time advancing visible to
ESP32 firmware without flooding the WS pipe with static
calibration dumps. Hash is primed during initial sync so the
first tick doesn't push a redundant identical buffer.
5. **Write-forwarding ProxySlave → peer**: backend ProxySlave
buffers write bytes during the transaction and emits a
`proxy_i2c_complete` event on STOP / repeated-START. Frontend
Esp32Bridge dispatches the event to a new onProxyI2cComplete
callback; the shim replays the byte sequence on the actual
peer I2CDevice via writeByte() + stop(). Makes ESP32 firmware
writes to peer SSD1306 actually repaint the OLED, peer PCF8574
latch updates, peer I2CMemoryDevice register mutations propagate.
6. **ESP32-C3 routed as bridge**: Interconnect.isBrowserSim no
longer claims c3/xiao-c3/c3-supermini — they were already
going through Esp32Bridge per the store's ESP32_RISCV_KINDS
routing, but Interconnect was treating them as browser sims
which broke proxy install. isEsp32Bridge now correctly
includes c3 family + ESP32-S3 + Arduino Nano ESP32.
Defensive: addBoard now disposes any existing shim's proxies
before overwriting simulatorMap entry so test reruns don't leak
timers.
Tests:
- 4 BFS multi-hop tests (i2c-multi-board-slave-gap.test.ts)
- 11 cross-board scenarios + per-peer + write-forward + resync
(i2c-esp32-multiboard-bridge.test.ts)
- 1 real-firmware E2E for write-forward via QEMU (compile +
load + observe proxy_i2c_complete arriving with the byte)
- New sketch fixture: esp32_i2c_write_to_peer.ino
Result: 90 test files / 1295 tests pass / 0 fail.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds the transactional email pipeline driven from the Odoo SMTP relay so
new sign-ups get a Velxio-branded welcome and existing users can reset a
forgotten password without us running our own outbound mail server.
Backend:
- PasswordResetToken model: one-time, SHA-256-hashed (plain text never on
disk), TTL 60 min, marked used_at on consume to prevent replay.
- POST /auth/forgot-password — anti-enumeration (always 200 + generic
message), rate-limited 3/hour/user.
- POST /auth/reset-password — verifies token, hashes new password,
atomically marks token used.
- /auth/register hooked with asyncio.create_task to fire welcome mail —
registration is never blocked on Odoo being up.
- New service app/services/odoo_mail.py: async httpx wrapper, fire-and-
forget, swallows every error so the request lifecycle stays clean.
- Settings ODOO_URL / ODOO_API_KEY / ODOO_MAIL_TIMEOUT_S /
PASSWORD_RESET_TOKEN_TTL_MINUTES / PASSWORD_RESET_RATE_LIMIT_PER_HOUR.
Frontend:
- /forgot-password page (single email field + "check your inbox" state).
- /reset-password?token=XYZ page (new password + confirmation, redirects
to /login?reset=ok on success).
- "Forgot your password?" link + green confirmation banner on /login.
- authService gains requestPasswordReset() and resetPassword().
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Implemented `i2c-esp32-real-firmware.test.ts` to test ESP32 I2C communication via backend and WebSocket.
- Created `load-example-transitions.test.ts` to ensure proper loading of examples between board-less and board-based contexts.
- Added `CircuitVerificationModal.tsx` to display circuit verification results before running simulations.
- Developed `circuitVerifier.ts` to perform pre-flight checks for circuit safety, identifying potential issues like short circuits and component overloads.
- Introduced minimal ESP32 I2C master sketch `esp32_i2c_writer.ino` for testing I2C transactions.
A user reported on Discord: "the Velxio Console doesn't update anything,
it just waits until the very end and displays everything in one go".
True for the async compile path — /compile/status only carried `state`
and the final `result`, so the editor's CompilationConsole stayed empty
during the 5-7 minute cold ESP-IDF builds and dumped 1500 lines at once
when the build finished.
This wires live build output through the whole stack.
Backend (espidf_compiler.py)
- New _run_with_streaming() helper. When a progress_callback is provided
it spawns the subprocess via Popen + stdout/stderr drain threads and
invokes the callback line-by-line. When None it falls back to the
existing subprocess.run(capture_output=True) one-shot path so the
unit-test code that doesn't care about live output is unaffected.
- compile() and _compile_in_dir() take an optional ProgressCallback.
- _run_cmake / _run_ninja closures now go through _run_with_streaming
with that callback. cmake configure (~2-5 s) + ninja (~5-300+ s) both
stream now; the ninja output is the one users actually want to watch.
Backend (compile.py)
- _compile_job seeds COMPILE_JOBS[id]['stdout_buffer'] = '' and defines
on_progress_line(line) which appends to it. Buffer capped at 256 KB
(tail kept) so a runaway build can't OOM the FastAPI process.
- The buffer is preserved on both the success and the error path so
late polls still see the log even after state transitions to
done/error.
- /compile/status now returns the buffer as a `stdout` field.
CompileStatusResponse gains the field with default '' so old clients
that don't read it still work.
Frontend (compilation.ts)
- compileCode() takes a 4th argument: optional CompileProgress
callback fired every poll while state ∈ {pending, running}. Carries
the cumulative stdout (caller computes deltas) plus elapsed seconds.
- Surfaces the new `stdout` field of /compile/status and forwards it
to the callback. Errors thrown from the callback are swallowed —
a faulty UI hook must never break the polling loop.
Frontend (EditorToolbar.tsx)
- Both compileCode() call sites (Run and Compile-All) now pass an
onProgress callback. It tracks `lastStreamedLen` per-compile, splits
each new delta on newlines, and appends them as `info`-typed
CompilationLog entries via setCompileLogs. The Compile-All flow
prefixes each line with the board label so multi-board builds stay
readable.
- After the build settles, the existing parseCompileResult call still
runs and appends the structured analysis on top of the live stream
— that's where FAILED-block detection + the `error`-typed entries
that drive the auto-switch-to-errors filter live.
Net effect on the user complaint: cold ESP-IDF builds now show the
ninja [N/1483] progress lines streaming into the console as they
happen, instead of staring at an empty panel for 5-7 minutes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three coordinated fixes that together close the "ESP-IDF compile takes
5-7 min every time" gap and prevent the failure mode where a user clicking
compile multiple times spawns six ninja processes that peel each other
apart on a modest VPS.
What was wrong
- /compile/start generated a fresh uuid4 every call, so 6 clicks = 6
independent builds racing each other. Saw load average 30 on the prod
VPS during a real BMP280 attempt today.
- No concurrency limit anywhere; asyncio.create_task() fired without
gating.
- ccache was wired in last week (PR #149) but reported 18,350 cacheable
calls and **0 hits** because the build dir was a fresh
tempfile.TemporaryDirectory(prefix='espidf_') per compile. The random
/tmp/espidf_<random>/ path baked into -I and -fmacro-prefix-map flags
→ different command line every compile → ccache hash miss every time.
What this PR does
1. Job deduplication (`backend/app/api/routes/compile.py`)
- New `_job_key(files, board_fqbn)` returns SHA-256 of normalised file
names + contents + board. Order-independent.
- New `JOB_BY_KEY: dict[str, str]` indexes hash → job_id.
- `compile_start` checks JOB_BY_KEY before spawning a new task; if a
job for this exact content is already pending or running, returns
the existing job_id (logs `[compile] dedup hit — reusing job <id>`).
- `_purge_expired_jobs` evicts both COMPILE_JOBS and JOB_BY_KEY,
keeping the index consistent. Edge case where two jobs share a key
(old finished, new running) is handled — only evict the key entry
if it still points at the purged job.
2. Concurrency control (`backend/app/api/routes/compile.py`)
- `_COMPILE_SEMAPHORE = asyncio.Semaphore(2)` global cap on
simultaneous compiles.
- `_target_lock(board_fqbn)` returns a per-target asyncio.Lock so
concurrent compiles to the SAME board (sharing the persistent build
dir) serialise. Different boards still run in parallel up to the
semaphore cap.
- `_compile_job` acquires sema → per-target lock → flips state to
`running` → calls `_run_compile`. Pending state now accurately
reflects "queued waiting for resources".
3. Persistent build dir (`backend/app/services/espidf_compiler.py`)
- New `_prepare_persistent_project_dir(idf_target)` materialises
`/var/lib/velxio-build/<target>/project/` from the template on
first use; on subsequent compiles it wipes only `main/` and
`user_libs/` (the per-compile parts) and leaves `build/` alone so
ninja's incremental cache + ccache .o files survive.
- Toolchain version sentinel (`.idf_version`) wipes the whole target
dir if the ESP-IDF or arduino-esp32 version changes — cached
objects from the old toolchain are no longer ABI-compatible.
- `compile()` is now a thin dispatcher: persistent path or fallback
to the legacy `tempfile.TemporaryDirectory()` flow. The actual
build logic was extracted into `_compile_in_dir()` so both paths
share one implementation, no duplication.
- Escape hatch: `VELXIO_PERSISTENT_BUILD_DIR=0` env var falls back
to the tempfile path without rebuilding the image. Critical for
production safety.
4. ccache normalisation (`Dockerfile.standalone`)
- + `ENV CCACHE_BASEDIR=/var/lib/velxio-build` makes ccache canonicalise
absolute paths under that prefix when computing the cache key.
Robustens hits against any future subdir rearrangement.
5. Docker compose (`docker-compose.yml`)
- + named volume `velxio-build:/var/lib/velxio-build` so the persistent
build dir survives `docker compose up -d --build`.
- + env `VELXIO_PERSISTENT_BUILD_DIR=1` (default ON; users disable
without rebuilding).
Expected impact
- Cold first compile per container per target: unchanged (~5-7 min).
- Same sketch re-compiled: ~2-5 s (everything cached).
- Different sketch, same target: ~5-30 s (only user code + new lib steps
rebuild; ESP-IDF base hits cache).
- Different sketch with new libraries: ~30-90 s (new lib component
compiles; rest hits cache).
- Concurrent clicks on same example: 1 build, others poll the same
job_id. No more six-ninja meltdown.
Tests
- `test/backend/unit/test_compile_dedup.py` covers `_job_key` stability +
variance and `_purge_expired_jobs` consistency (including the
"two jobs share a key" edge case).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Cold first compile per container is unchanged (cache empty). Subsequent
compiles drop from ~5-7 minutes to ~30-60 seconds because every ESP-IDF
base object (FreeRTOS, lwIP, esp_wifi, libsodium, soc, hal, …) hits the
cache. The user's BMP280 example, which hangs on cold compile, completes
near-instantly on the second attempt.
Why a transparent cache is safe: ccache hashes the preprocessed source +
flags + compiler. A cache hit only happens when the input is byte-for-byte
identical to a prior compile. Different sketches with different libraries
still get correct cache misses; there is no path where one project's
output contaminates another.
Changes
- Dockerfile.standalone: install ccache, set CCACHE_DIR=/var/cache/ccache,
IDF_CCACHE_ENABLE=1, configure 2 GB cap with compression. Compression
(level 6) cuts cache disk usage by ~40% with negligible CPU overhead.
- docker-compose.yml: named volume `ccache:/var/cache/ccache` so the
cache survives `docker compose up -d --build` (without it, every image
rebuild discards the cache).
- backend/app/services/espidf_compiler.py: pass `-DCCACHE_ENABLE=1` to
cmake when IDF_CCACHE_ENABLE is truthy. ESP-IDF's project.cmake
(`set_property(GLOBAL PROPERTY RULE_LAUNCH_COMPILE ccache)` on line 374)
is what actually wires ccache in; without the cmake -D flag the env
var alone has no effect because we don't go through idf.py.
Escape hatch: set IDF_CCACHE_ENABLE=0 in compose env to disable without
rebuilding the image.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The synchronous /api/compile endpoint forced one long-lived HTTP request
to span the entire build. Cloudflare's 100s edge timeout cuts that off
mid-flight for any cold ESP-IDF compile (BMP280 takes 5-7 min on first
run). The user-visible symptom was HTTP 524 well before the backend
even noticed.
Backend (compile.py)
- New `POST /api/compile/start` returns `{job_id}` immediately and
spawns the actual compile as an asyncio.create_task background.
- New `GET /api/compile/status/{job_id}` returns the current job state
(`pending` | `running` | `done` | `error`). Each poll completes in
milliseconds, far under any edge timeout.
- Existing `POST /api/compile/` kept verbatim for backward compatibility
(AVR/RP2040 builds finish in seconds and don't trip 524).
- Build logic extracted into `_run_compile()` so both paths share one
implementation; no duplicated ESP-IDF / arduino-cli branching.
- Async path opens its own short-lived DB session via AsyncSessionLocal
for metric recording — the request-scoped session is dead by the time
the background task finishes.
- COMPILE_JOBS dict purges entries 30 minutes after completion so a
busy server doesn't grow unboundedly.
Frontend (compilation.ts)
- compileCode() now: POST /compile/start → poll /compile/status every 2s
until state ∈ {done, error}, with a 15-minute client-side cap.
- 30s axios timeout per individual call (not per build) so transient
network blips during a long compile auto-retry instead of failing.
- 404 on /status throws (job expired / server restarted); other poll
errors warn and retry. Surfaces structured error responses verbatim
so the editor's compile-error panel keeps working unchanged.
Limitation: COMPILE_JOBS lives in-process; if velxio ever scales to
multiple FastAPI workers this needs to move to Redis or sqlite. Single-
instance is fine today.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The ESP32 BMP280 example compile was timing out at 98% (1473/1483 build
steps), failing with the unhelpful "ESP-IDF build timed out (300s)"
message even though every individual step was healthy.
Cold ESP-IDF builds that pull in external Arduino libraries — Adafruit
BMP280 + Adafruit BusIO + Adafruit Unified Sensor on top of the base
arduino-esp32 component tree — routinely produce ~1480 build objects.
On modest VPS hardware this takes 5-7 minutes the first time. Ninja's
incremental cache makes subsequent compiles seconds, but the first one
needs more headroom.
Constant lifted to NINJA_TIMEOUT_S so the value used in the timeout
matches the value reported in the error message — the previous code
hard-coded "300s" in two places that were free to drift apart.
Repro before: open the example "ESP32 — BMP280 Barometric Pressure"
on velxio.dev/editor on a clean container, click compile → fails after
5 minutes with timeout. After: completes in ~6 minutes on the first
run, ~5 seconds on subsequent runs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI's e2e test_hcsr04_simulation.mjs caught the regression introduced by
a3f21a2 (the issue #101 fix). The worker crashes on boot with
Firmware decode error: No module named 'app'
esp32_worker.py runs as a subprocess via subprocess.Popen([sys.executable,
WORKER_PATH, ...]). When Python launches a script directly, sys.path[0]
is the SCRIPT's directory (backend/app/services/), not the backend root.
So `from app.services.esp32_flash_image import pad_to_flash_size` fails
because there is no `app/` under `backend/app/services/`.
esp32_lib_bridge.py wasn't affected because it runs in-process inside
uvicorn, where backend/ is implicitly on sys.path.
Fix: same try/except + importlib fallback the worker already uses for
esp32_i2c_slaves at the top of the file. First try the package import
(works when imported by the bridge's tests or anything else with the
backend root on sys.path), fall back to direct file loading otherwise.
Verified the fallback works in isolation by simulating the subprocess
context (sys.path containing only backend/app/services/) — the package
import fails as expected and the file-load fallback returns a properly
padded 4 MB buffer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
A user reported the LEDC PWM RGB example failing to compile. The sketch
calls ledcAttach(pin, freq, resolution) — the one-shot API added in
arduino-esp32 3.x. Our toolchain image pins arduino-esp32 to 2.0.17
(matched to ESP-IDF 4.4.7 + the lcgamboa QEMU ROM), where the API is
the older two-step ledcSetup + ledcAttachPin pair. Sketches written
against 3.x docs hit "ledcAttach was not declared in this scope".
Bumping arduino-esp32 to 3.x means moving to ESP-IDF 5.x, which may
break our QEMU fork. Cheaper fix: ship a compat shim header in the
ESP-IDF project template that defines ledcAttach + ledcAttachChannel
in terms of the 2.x API, gated on `!defined(ledcAttach)` so it
disappears the day we bump.
espidf_compiler.py now injects #include "velxio_compat.h" right after
Arduino.h whether the user explicitly included Arduino.h or we
prepended it ourselves.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two reports rolled into one sdkconfig change:
1. Beta testers reported "weird serial output" across DHT22, Servo+Pot,
Joystick, WiFi×2, and DualADC examples. The user's Serial.print lines
came interleaved with internal ESP-IDF chatter:
I (53306) gpio: GPIO[4]| InputEn: 1| OutputEn: 0| ...
I (10626) phy_init: phy_version 4791,2c4672b,...
sdkconfig.defaults didn't pin a default log level so it inherited
CONFIG_LOG_DEFAULT_LEVEL_INFO. Set WARN (level 2) so only warnings
and errors leak into the user's serial output. Sketches can still
esp_log_level_set() per tag at runtime if they want verbose.
2. BLE Advertise example failed to compile. sdkconfig had NimBLE
enabled, but arduino-esp32 2.0.17's BLEDevice.h targets Bluedroid.
Switch the stack: enable BT_BLUEDROID + BTDM_CTRL_MODE_BR_EDR_BLE +
BT_BLE so the standard arduino-esp32 BLE library compiles. Sketches
that explicitly include NimBLEDevice.h will not compile under this
config — that's a smaller minority than the BLEDevice.h users.
The runtime side of BLE still depends on stubs in the qemu-lcgamboa
fork; this only fixes the compile path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Issue #101 reproducer: an ESP32 sketch that pulls in Adafruit_SSD1306 +
Adafruit_GFX produced a "No response from server. Is the backend
running on port 8001?" error in the browser. The compile actually
succeeded backend-side, but the JSON response carrying the firmware
was ~5.5 MB of base64 — the ESP-IDF compiler builds a full 4 MB merged
flash image (mostly 0xFF padding), encodes it whole, and ships it. In
prod that response goes through nginx + Cloudflare, which buffer-fail
or RST the connection on payloads that big — axios then lands in the
"no response" branch with no HTTP status to surface.
Fix: trim the trailing 0xFF padding before serializing, re-pad to a
valid QEMU flash size (2/4/8/16 MB) just before mtd attach. Lossless:
bytes after `last_used` in the merge are 0xFF by construction, so
trim → pad reproduces the original image byte-for-byte.
Numbers from the reproducer (Adafruit_SSD1306 + Adafruit_GFX,
esp32:esp32:esp32 board):
before: ~5.5 MB JSON response
after: 539 KB JSON response (10× smaller)
backend/app/services/espidf_compiler.py
_merge_flash_image now tracks `last_used` across the three placed
sections (bootloader / partitions / app) and writes only
flash[:last_used] to merged_flash.bin.
backend/app/services/esp32_flash_image.py (new)
Shared `pad_to_flash_size(bytes) -> bytes` helper. Rounds up to the
next valid QEMU flash size with a 4 MB minimum, matches the
frontend's existing padToFlashSize logic in Esp32MicroPythonLoader.
Raises ValueError on >16 MB inputs (would indicate a broken upstream
merge, not anything user-recoverable).
backend/app/services/esp32_lib_bridge.py
backend/app/services/esp32_worker.py
Both QEMU consumer paths (in-process and subprocess) call
pad_to_flash_size right after base64.b64decode, before writing the
tmp .bin that QEMU attaches with `-drive if=mtd,format=raw`.
Verified: smoke test confirms trim → pad → original is byte-exact.
Edge cases covered: small payloads pad up to the 4 MB minimum;
firmwares >16 MB are rejected loudly.
Closes#101
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds 4 nullable columns to the users table so deployments that wire an
external billing system (e.g. velxio.dev with Odoo) have a place to cache
subscription status. Self-hosters never write these — defaults are safe
(no paid features unlock).
- models/user.py: is_paid_subscriber (bool, default false), subscription_status
(str|None), subscription_period_end (datetime|None), odoo_partner_id
(int|None, indexed).
- main.py: 4 ALTER TABLE statements appended to the legacy_migrations list
so existing deployments auto-migrate on next boot.
- schemas/auth.py: extend UserResponse with the 3 user-visible fields
(is_paid_subscriber, subscription_status, subscription_period_end). The
frontend useAuthStore already persists the whole UserResponse, so these
surface automatically without any frontend changes upstream.
odoo_partner_id stays internal — clients don't need it.
Zero behavioural change for existing OSS deployments.
Three small, backwards-compatible hooks let anyone with private features
(velxio.dev's analytics, custom integrations, paid tiers, …) layer them
on top of the open-source build without forking files.
Backend (app/main.py):
- After standard router registration, try-import an optional `app.pro`
module exposing `register_pro(app)`. ImportError is silently swallowed
(the OSS image doesn't ship `app.pro`, so this is a no-op there).
Frontend:
- EditorToolbar: new optional `rightSlot` prop renders extra elements
after the built-in right-group buttons (mirrors the existing
`centerSlot` pattern).
- main.tsx: dynamic `import('@pro/index')` gated by VITE_PRO_BUILD env.
When unset (OSS build), the branch is dead-code-eliminated and no pro
chunk is emitted.
- vite.config.ts: `@pro` alias resolves to `src/__pro_stub__/` by default.
Private builds set `VITE_PRO_BUILD=true` and `PRO_OVERLAY_PATH=<path>`
to point at their real overlay tree.
- src/__pro_stub__/index.ts: 1-line no-op `mountPro` so TypeScript and
Vite resolvers stay happy in OSS builds.
Verified: `npm run build:docker` succeeds; `npm test` passes 1161/1162;
the OSS bundle (43 MB) contains zero references to `__pro_stub__`,
`@pro`, or `pro/index` (verified via `grep dist/`).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The _should_include filter inside _merge_arduino_libs_to_component
rejected every subdirectory of a library except 'utility/' when the
library lacked a src/ layout. That blocked legitimate header dirs like
Adafruit_GFX_Library/Fonts/, breaking compiles that use any GxEPD2
example with a custom font (#include <Fonts/FreeMonoBold12pt7b.h>).
The earlier filter at the top of the function already excludes
docs/examples/tests/etc. via excluded_dirs, so anything that survives
that check is presumed to be buildable source. Letting all remaining
subdirs through restores Fonts/, gfxfont/, and similar conventional
auxiliary header directories that Adafruit-style libs rely on.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The transitive-include scan in _merge_arduino_libs_to_component used
glob('*.h') on the component dir, which only finds headers at the root.
Libraries with a src/ layout (GxEPD2, ArduinoJson, most modern Arduino
libs) keep their headers under src/<...>, so the scan saw zero headers
and never queued their transitive deps.
Symptom: compiling a sketch that includes GxEPD2_3C.h failed with
'Adafruit_GFX.h: No such file or directory' even though Adafruit_GFX
was installed via the Library Manager — because the BFS never reached
its header from inside GxEPD2_GFX.h.
Switching to rglob('*.h') walks the full directory tree and lets the
BFS pick up Adafruit_GFX, Adafruit_BusIO, and any other transitive
dependency that lives under src/.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The directory grew well beyond Wokwi-only contents: it now hosts
lcgamboa's QEMU fork (qemu-lcgamboa), Espressif's esp32-camera, the
ngspice WASM build, fritzing-parts, picowi, an alternative QEMU
(qemu-esp32), the 100_Days_100_IoT_Projects examples repo, and
Wokwi's own avr8js/rp2040js/wokwi-elements/wokwi-features/wokwi-boards.
"wokwi-libs" was misleading — half the contents have nothing to do
with Wokwi. "third-party/" is the standard convention for vendored
external dependencies.
Mechanical changes:
Path rename:
wokwi-libs/ → third-party/
update-wokwi-libs.bat → update-third-party.bat
docs/WOKWI_LIBS.md → docs/THIRD_PARTY.md
Submodule reconfiguration:
.gitmodules — 4 path= and section names updated
.git/modules/wokwi-libs/ → .git/modules/third-party/
each submodule's .git file rewired to ../../.git/modules/third-party/<name>
Reference updates (~80 files): vite.config.ts aliases, Dockerfile
COPY paths, GH Actions workflow steps, build_qemu_*.sh, all
docs/* and test/*/autosearch/* entries that mention the path,
package-lock.json file: dependencies, .gitignore patterns,
sitemap.xml + index.html SEO blurbs, scripts/generate-component-*,
.dockerignore, .idea/vcs.xml. Bulk replaced both `wokwi-libs/`
(path) and bare `wokwi-libs` (textual mentions in docs/comments).
Verified:
- npx tsc -b --noEmit produces no new errors related to these paths
- vite.config.ts aliases now point at ../third-party/avr8js etc.
- All 4 git submodules (avr8js, rp2040js, wokwi-elements,
wokwi-features) are linked under third-party/ with their
worktrees re-populated and config files referencing the new path
- `grep -r wokwi-libs` returns zero hits outside node_modules,
.vite, frontend/dist, third-party/ (upstream submodule contents),
*.pyc caches, and *.dll.pre-camera rollback binaries
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User reported only 2 frames rendering after the SPI batching change.
Diagnosis: the previous flush triggers (CS-line HIGH, buffer >=4096)
were both event-driven. Adafruit_ILI9341's ESP32 backend manages CS
via digitalWrite — i.e. through the GPIO peripheral, NOT the SPI
peripheral's hardware CS pin. So the CS-line-HIGH event from
picsimlab_spi_event NEVER fires for this driver. The buffer only
flushes when it hits 4096 bytes.
Frame 1: 38 400 bytes from drawRGBBitmap → 9 flushes at 4096-byte
boundaries → last 192 bytes stay in the buffer. Status bar adds
some bytes too → maybe one more flush.
Frame 2: same. But by frame 3 the firmware is running ahead of the
flush rhythm and somehow the buffer pattern wedges in a state where
no flush completes (likely a partial buffer that sits between
transactions while the firmware briefly waits on the next fb_get).
Hard to reproduce deterministically — but the symptom matches.
Fix: add a 50 ms periodic flush thread. Independent of any event,
it acquires the lock and flushes whatever's pending. Bounds the
worst-case latency at 50 ms (= 20 fps ceiling, more than enough for
the emulator).
Triple-trigger now:
1. CS HIGH (fast path for hardware-CS drivers)
2. Buffer >= 4096 (safety for big transactions)
3. 50 ms timer (catches GPIO-CS drivers, prevents stalls)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User reported the ESP32-CAM + ILI9341 live preview at ~1 frame/min.
Profile: 80×60 preview pushes 9600 SPI bytes per drawRGBBitmap, and
each byte was emitting a full {type:'spi_event'} JSON message over
the worker→backend→WS→frontend pipeline. Per-byte overhead ~150-200µs
in Python (json.dumps + sys.stdout.write+flush dominates) plus
asyncio + WS dispatch. Net: 1.5-2 sec/frame minimum, much worse with
GIL contention.
Fix: buffer MOSI bytes in the worker and emit a single base64-encoded
`spi_batch` message when CS goes HIGH (transaction ended) or the
buffer crosses 4 KiB. ~9600 events/frame collapse to ~3 messages.
backend/app/services/esp32_worker.py:_on_spi_event
- Add _spi_byte_buf bytearray + threading.Lock
- On op==0x00 (byte): append; flush early if buf >= 4096
- On op==0x01 (CS change): flush buffer, then emit the CS event
via the legacy spi_event channel (ePaper / custom chips that
observe CS still get it).
frontend/src/simulation/Esp32Bridge.ts
- New 'spi_batch' message handler decodes b64 and replays each
byte through the existing onSpiByte callback. Parts that
subscribed via simulator.spi.onByte don't notice the protocol
change. The 'spi_event' branch still handles CS changes plus
legacy single-byte payloads for backwards compat.
Now that 38 KB/frame is cheap, restore preview to 160×120 + JPEG
quality 0.35 in the gallery example. Real measured speedup: ~50× on
the QVGA preview demo. Real hardware was never affected — it runs
SPI at 80 MHz and pushes the bitmap in ~4 ms either way.
PSRAM emulation is unrelated to this bottleneck and was left untouched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User confirmed the FULL pipeline works with their laptop webcam at
QVGA quality 0.6 — frontend → WS → backend → worker → DLL → I²S →
firmware → esp_camera_fb_get() → user sketch.
[Frame #1] 8192 bytes 320x240 fmt=4
├─ SOI (FF D8 FF): ✓ at offset 0
├─ EOI (FF D9): ✓ at offset 8190
└─ First 16 bytes: FF D8 FF E0 00 10 4A 46 49 46 …
(JFIF, real webcam)
Stats: 10/10 Valid JPEGs at ~3.5 fps.
Changes:
- wokwi-libs/qemu-lcgamboa @ e4321d1 (picsimlab-esp32):
EOFS_PER_FRAME 6→8, inject_eoi_now flag for EOI injection on the
last EOF of each VSYNC burst. Handles JPEGs of arbitrary size by
forcing FF D9 at offset 8190 — JPEG decoders tolerate the
truncation gracefully.
- backend/esp32_worker.py: throttled trace log every 30 frames
(`camera_frame #N received (NNNN bytes)`) so users can confirm
the frontend → worker leg is alive without flooding the log.
- test/test-esp32-cam/autosearch/14: documented bug #9 (the 9th and
final silent bug — real webcam JPEGs exceed the deliverable byte
budget) with full forensic trace + final architecture diagram.
The emulation now handles ANY user webcam → ESP32-CAM use case end
to end. Standard upstream esp_camera_init() / esp_camera_fb_get()
sketches work without modification.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First open-source end-to-end emulation of the AI-Thinker ESP32-CAM
in QEMU, paired with a browser webcam → firmware bridge so users can
develop camera sketches without hardware. Status: esp_camera_init()
returns ESP_OK; OV2640 chip-id verifies (PID/VER/MIDH/MIDL exactly
match the datasheet); GPIO 25 VSYNC NEGEDGE interrupt enabled by
the upstream driver. Final piece (cam_task accepting frames) is in
progress — descriptor walker fix landed in this commit.
Backend (Python/FastAPI):
- simulation.py: camera_attach/frame/detach WS handlers
- esp32_worker.py: ctypes binding to velxio_push_camera_frame +
feature-detection fallback for older DLLs
- esp32_lib_manager.py: forward camera commands to the worker stdin
- esp-idf-template/main/CMakeLists.txt: esp32-camera headers added
via add_prebuilt_library + REQUIRES driver (resolves i2c_master_*
symbols). LED_BUILTIN=2 fallback for sketches that hardcode it.
Frontend (React/TS):
- EditorToolbar.tsx: ESP32-CAM (and the rest of the ESP32 family)
added to isQemuBoard list — Run button now starts the QEMU bridge
for these boards instead of falling through to the AVR path
- useWebcamFrames.ts: getUserMedia → OffscreenCanvas →
toBlob('image/jpeg') → base64 → WS at ~10 fps
- CameraToggle.tsx: header button with status colors + frame counter
- SimulatorCanvas.tsx: render CameraToggle for esp32-cam boards
- Esp32Bridge.ts: sendCameraAttach/Frame/Detach + chunked btoa
- useSimulatorStore.ts: diagnostic log on compileBoardProgram
- components-metadata.json: regen including esp32-cam component
Submodule pointer:
- wokwi-libs/qemu-lcgamboa → ff8eee0 (camera devices commit on
davidmonterocrespo24/qemu-lcgamboa branch picsimlab-esp32)
Investigation + tests in test/test-esp32-cam/:
- 13 autosearch markdown docs (overview, SOTA, OV2640 spec, DVP/I2S
spec, build blueprint, blockers resolved, descriptor walker fix)
- 5 sketches (camera_init, sccb_probe, dma_smoke, frame_roundtrip,
webcam_demo) + 8 live + WS regression tests
- README with the user-facing flow
.gitignore:
- libqemu-*.dll.{pre-camera,new,bak} (rollback points, regenerated)
- wokwi-libs/esp32-camera/ (clone consumed by arduino-esp32 path,
not part of this repo)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Backend:
- Add version field to InstallLibraryRequest
- Add fallback and requested_version to InstallResponse
- Add DELETE /api/libraries/uninstall endpoint
- Enhance install_library() for versioned installs (LibName@version)
- Add semver validation and fallback logic
- Add uninstall_library() method
- Fix _parse_version() to reject non-numeric version parts
Frontend:
- Update installLibrary() with optional version parameter
- Add uninstallLibrary() and resolveLibraryVersion() helpers
- Add version selector dropdown in Library Manager
- Add UNINSTALL button for installed libraries
- Show fallback messages when requested version unavailable
- Add parseLibSpec() and version badges in InstallLibrariesModal
_create_idf_component was fixed but NOT _merge_arduino_libs_to_component.
The latter was still flattening library files, causing ArduinoJson compilation to fail
with 'src/ArduinoJson.h: No such file or directory'.
Changes:
- Replace flat copy with directory-structure-preserving copy
- Add exclusion logic for non-buildable directories (examples, tests, docs)
- Generate INCLUDE_DIRS from actual directory structure
- Track files by relative path to prevent name collisions
This ensures libraries with src/ layouts (ArduinoJson, etc.) compile correctly.
- Maintain original library layout (src/, utility/) instead of flattening files
- Add exclusion logic for non-buildable directories (examples, tests, docs, CI)
- Dynamically generate INCLUDE_DIRS from actual directory structure
- Add validation to ensure buildable source files exist before proceeding
- Support both flat and src-based library layouts
- Fix path separator normalization for cross-platform compatibility
This improves compatibility with complex Arduino libraries that rely on
specific directory structures and relative includes.
The project save/load pipeline only persisted a single `board_type`, so
multi-board workspaces silently lost every board except the active one
on save, and wires referencing the dropped boards' IDs orphaned to the
canvas corner on reload. An audit of the production backup found 74/306
projects (24%) with at least one orphaned wire and 174/301 non-trivial
projects whose code was still the default Blink template — strong signal
that users save once and never re-save.
Backend
- Add `boards_json` column on `projects` with idempotent ALTER TABLE in
the lifespan migration list.
- New `FileGroup` schema + `file_groups` array on
ProjectCreate/Update/Response. Legacy `files`/`code` kept for back-compat.
- `project_files.py` now uses `{pid}/{groupId}/{filename}` subdirs via
`read_groups`/`write_groups`. Legacy flat layouts are auto-promoted on
read; legacy single-list `files` only updates the active group, leaving
other boards' files intact.
- `_persist_files_from_body` honors file_groups → files → code priority.
Frontend
- `useSimulatorStore.addBoard` accepts an optional `explicitId` so
saved board IDs can be restored verbatim (wires reference IDs literally).
- New `loadProjectState({boards, fileGroups, components, wires,
activeBoardId})` action: tears down current boards, recreates from the
payload, restores file groups atomically, recalculates wire positions
on the next frame, and refreshes the Interconnect.
- `useEditorStore.replaceFileGroups` for atomic multi-group restore.
- `SaveProjectModal` and `ProjectByIdPage`/`ProjectPage` now go through
`buildSavePayload` / `buildLoadPayload` (handles pre-backfill projects
by synthesising a default board from `board_type`).
Auto-save (#useAutoSaveProject hook)
- 2.5s debounced silent PUT triggered ONLY when an authenticated user
has a `currentProject` with a UUID. State hash detects real changes
vs. UI-only churn; baseline is reset on project load so the just-loaded
state isn't immediately re-saved.
- `beforeunload` flush via `fetch keepalive: true` (supports PUT +
credentials, survives unload).
- Compact status indicator in `AppHeader` (idle/dirty/saving/saved/error).
Backfill script (one-off, idempotent)
- `backend/scripts/backfill_boards_2026_05.py` populates `boards_json`
for legacy projects. Heuristic per project, based on which board IDs
the wires reference:
Case A — wires only ref 'arduino-uno' but board_type ≠ uno:
rename id→board_type and rewrite wire endpoints.
Case B — single-board normal: keep verbatim.
Case C — multi-board: recreate one board per distinct ref, infer
kind by stripping trailing -N suffix.
Also moves any flat files into the active board's group subdir.
Stdlib-only, runs from host or `docker exec`.
Docker
- `Dockerfile.standalone` now copies `backend/scripts/` into the image
so the backfill is callable via `docker exec velxio-app python
/app/scripts/backfill_boards_2026_05.py --apply`.
Verified locally on the restored production backup (363 projects):
33 Case A, 316 Case B, 14 Case C, 135 wire endpoints renamed, 0 orphans.
Re-running the script after apply skips all 363 (idempotent).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Update esp32_lib_manager.py to dynamically set library extensions based on the platform (Linux, Windows, macOS).
- Update submodule references for qemu-lcgamboa and wokwi-elements.
- Add documentation on ESP32 Arduino runtime crashes related to cache disable during WiFi/BT initialization.
- Introduce regression tests for the ESP32-CAM blink issue, ensuring the user sketch matches the reported problem.
- Implement an IRAM-safe blink sketch to confirm the regression is due to the Arduino runtime.
- Create a comprehensive test suite to cover various layers of the simulation and compilation process.
- Implement UC8159cDecoder for handling 7-colour ACeP panels.
- Introduce painting functions for UC8159c frames in EPaperPart.
- Update EPaperPart to handle both SSD168x and UC8159c frame types.
- Add integration tests for EPaperPart and UC8159cDecoder.
- Create example sketch for 5.65" ACeP 7-colour panel.
- Enhance error handling in test cases for library dependencies.
- Introduced EPaperPanels.ts to define configurations for various ePaper panels including dimensions, refresh rates, and controller details.
- Implemented SSD168xDecoder.ts to handle the decoding of SPI commands for the SSD168x family of ePaper displays.
- Created EPaperPart.ts to manage the simulation of ePaper panels, integrating with the existing simulator architecture and handling events.
- Added example sketches for 2.13", 2.9", 4.2", and 7.5" ePaper displays, demonstrating basic functionality and text rendering.
- Ensured compatibility with AVR, RP2040, and ESP32 platforms, with appropriate pin configurations for each.
- Added tcp_nat.py to handle TCP NAT, implementing the three-way handshake, data flow, and connection state management.
- Introduced udp_nat.py for UDP NAT, managing chip-initiated datagrams and maintaining flow state.
- Created integration tests in test_picow_net_bridge.py to validate the functionality of the TCP and UDP NAT implementations, including ARP, DHCP, ICMP, and DNS interactions.
- Introduced SVG layout dimensions for Phase 1 (B/W mono) and Phase 2 (colour) ePaper panels, detailing active areas, bezels, and pin layouts.
- Developed a phased emulation plan outlining the architecture and deliverables for different panel types, including SSD168x and UC81xx.
- Created a canonical "Hello, World!" sketch for the 1.54" ePaper panel, ensuring compatibility across ESP32, Raspberry Pi Pico, and Arduino Uno.
- Implemented a pure Python SSD168x decoder to validate SPI command sets and framebuffers against specifications.
- Added tests for compiling the hello-world sketch across supported boards and for the SSD168x protocol to ensure correct framebuffer behavior.
- Implemented handshake tests to validate initial bus state and register responses.
- Created end-to-end tests for Pico W LED blinking using MicroPython firmware.
- Added SDPCM framing tests to ensure proper encoding and decoding of control frames.
- Developed IOCTL tests to verify command responses and state changes in the emulator.
- Established a full lifecycle test for WiFi operations, including scanning, connecting, and packet handling.
- Introduced TypeScript configuration for test files to ensure compatibility and strict type checking.
- Implemented `esp32_spi_chip_demo.ino` to demonstrate SPI communication with a 74HC595 shift register.
- Created `esp32_uart_chip_demo.ino` for UART loopback testing with ROT13 transformation.
- Added Python tests for compiling chips and sketches, ensuring valid WASM output and successful compilation for various board families.
- Developed end-to-end tests for ESP32 with custom chips using I2C and SPI, validating synchronous communication through the backend.
- Introduced GPIO bridge tests to verify serial communication and GPIO state changes.
- Ensured all tests validate the expected behavior of the custom chips and their interaction with the ESP32 firmware.
Track per-user, per-project, and per-board usage to inform pricing tier
decisions. Adds an admin dashboard with KPIs (DAU/WAU/MAU, totals,
success rate), time-series charts for compiles/runs, board family +
FQBN breakdowns, "board diversity" pie chart (key freemium signal),
top users/projects, and per-country breakdown via Cloudflare's
CF-IPCountry header. Admin can now also view private projects.
Backend:
- New UsageEvent table (append-only event log with user_id, project_id,
event_type, board_fqbn/family, country, error_kind, duration_ms)
- Aggregate counters on User (total_compiles/runs/errors, last_active,
signup_country, last_country) and Project (compile/run/update counts,
last_compiled/run timestamps) kept in sync by MetricsService for O(1)
dashboard reads
- 10 admin endpoints under /api/admin/metrics/{overview, timeseries,
boards, board-diversity, top-users, top-projects, countries,
users/{id}, projects/{id}}
- POST /api/metrics/run for client-side run telemetry
- Country detection via cf-ipcountry header (no DB / no API calls)
- Auto-migrations in lifespan for legacy DBs
Frontend:
- recharts-powered Dashboard tab with KPI cards and 4 charts
- New Boards tab with per-family + per-FQBN breakdown
- Country column with flag emoji on Users tab
- Top countries card on Dashboard
- compileCode now forwards project_id; Run button reports via WS
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Implement `ammeter-waveform.test.ts` to validate AC readings from a sine wave source.
- Create `capacitor-charge-transient.test.ts` to test the charging response of an RC circuit driven by a microcontroller pin.
- Introduce `esp32-rectifier-integration.test.ts` for testing rectifier behavior using QEMU and ESP32.
- Add helper functions in `esp32RectifierE2E.ts` for the rectifier test harness.
- Develop `voltmeter-waveform.test.ts` to ensure correct AC and DC readings from a sine wave source.
- Implement unit tests for waveform statistics in `waveform-stats.test.ts` to validate RMS, mean, peak, and interpolation functions.
- Create `waveformStats.ts` to provide statistical functions for time-domain waveform analysis.
- Implemented Bmp280Element as a custom web component for the BMP280 barometric sensor, including SVG representation and pin configuration.
- Created CircuitPreview component to render circuit thumbnails using SVGs of components, including support for various boards and components.
- Added a script to generate SVG files from wokwi-elements, ensuring proper formatting and structure for reliable rendering.
- Introduced a test HTML generation script to visualize component SVGs.
refactor: Update wiring examples for E32 OLED integration; correct pin mappings for VCC, GND, DATA, and CLK
test: Improve unit tests for ESPIDFCompiler; add scenarios for library resolution and CMake patching
chore: Mark subproject commits as dirty for wokwi-libs
This commit introduces a detailed markdown document outlining the process of simulating DHT22 and HC-SR04 sensors on the ESP32 platform using Velxio's QEMU fork. The documentation covers the context of the simulation, key callbacks, problems encountered, and solutions implemented for both sensors. It includes architectural details, end-to-end testing procedures, and guidelines for adding new GPIO-timed sensors. The aim is to provide maintainers with a thorough understanding of the GPIO logic and the challenges faced during development.
- Updated the threshold for switching to data mode in MPU6050Slave from 2 to 3 WHO_AM_I reads to ensure correct chip identification.
- Enhanced comments in the code to clarify the sequence of I2C events during initialization.
- Added a new test file `test_mpu6050_emulation.py` to validate the MPU6050Slave state machine and ensure it handles the full Adafruit_MPU6050::begin() event sequence correctly.
- Updated existing tests to reflect the changes in the I2C handling logic.
- Modified `components-metadata.json` to update the generated timestamp.
- Marked submodules `rp2040js` and `wokwi-elements` as dirty to reflect local changes.
- Introduced I2C_SENSOR_MAP for pre-registering I2C sensors in the simulator store.
- Implemented I2C slave state machines for MPU6050, BMP280, DS1307, and DS3231 sensors in esp32_i2c_slaves.py.
- Added unit tests for I2C slave functionality covering BMP280, DS1307, DS3231, and I2CWriteSink.
- Updated the simulator store to handle I2C address resolution and sensor data management.
- Marked submodules as dirty in wokwi-libs for rp2040js and wokwi-elements.
- Implemented _MPU6050Slave and _BMP280Slave classes for I2C communication.
- Enhanced main function to register these sensors and handle I2C events.
- Updated sensor management to support MPU-6050, BMP280, DS1307, DS3231, SSD1306, and PCF8574.
- Added frontend examples for BMP280 weather station and SSD1306 OLED display.
- Modified Esp32Bridge to handle new I2C transaction events.
- Updated ProtocolParts to support ESP32 path for I2C devices.
- Enhanced useSimulatorStore to manage I2C transaction listeners.
Two fixes for ESP32 WiFi not connecting in production:
1. espidf_compiler.py: Change WiFi normalization from 'Velxio-GUEST' on
channel 6 to 'Espressif' on channel 5. The lcgamboa QEMU binary
downloaded from GitHub Releases only contains the original three APs:
PICSimLabWifi (ch1), Espressif (ch5), MasseyWifi (ch10). Channel 6
had no matching AP, so the beacon timer's channel-match condition never
fired → firmware scanned forever and never connected.
2. esp32_worker.py: Redirect fd 1 to /dev/null before loading QEMU so
raw UART bytes from QEMU's -nographic mux don't corrupt the JSON
event pipe. The real pipe fd is saved and sys.stdout is rebound so
_emit() continues to work. This also prevents stdout pipe back-pressure
from stalling qemu_main_loop() (and thus REALTIME timers).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Frontend WiFi detection via file-content scanning was unreliable because
fileGroups[board.activeFileGroupId] could be an empty array (not null),
bypassing the ?? fallback to editorState.files.
Fix: the ESP-IDF compiler now returns has_wifi:bool in its compile response.
The frontend stores this on the BoardInstance and uses it in startBoard()
instead of scanning file contents. The file-content scan is kept as a
fallback for boards that haven't been compiled in this session.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
_build_env() on Linux only set IDF_TOOLS_PATH but never added the tool
binary directories to PATH, so cmake could not find riscv32-esp-elf-g++
when compiling for ESP32-C3. Also improve ninja failure logging to show
stdout (where build errors actually appear) instead of empty stderr.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Dockerfile: download pre-built .so + ROM from velxio public release
instead of building from private qemu-lcgamboa source
- espidf_compiler: normalize any WiFi SSID → "Velxio-GUEST" for QEMU
compatibility (channel 6, open auth)
- docker-compose.yml: unified dev/prod using Dockerfile.standalone
- .dockerignore: exclude qemu-lcgamboa source from Docker context
- .gitignore: ignore prebuilt/ binaries, keep .gitkeep
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace arduino-cli with ESP-IDF 4.4.7 for ESP32 compilation — Arduino-compiled
firmware crashes in QEMU (9-28 reboots) while ESP-IDF boots cleanly (0 reboots).
The new espidf_compiler translates Arduino WiFi/WebServer sketches to native
ESP-IDF C code, compiles with cmake+ninja, and merges into 4MB flash images.
Key changes:
- ESP-IDF compiler: translates WiFi.begin/WebServer to esp_wifi/esp_http_server
- ESP-IDF project template with QEMU-optimized sdkconfig (DIO, 40MHz, no WDT)
- WiFi status parser for ESP-IDF serial logs (wifi_status, ble_status events)
- IoT Gateway HTTP reverse proxy for ESP32 web servers
- WiFi/BLE auto-detection from sketch content + visual status icons
- Static IP 192.168.4.15 matching slirp DHCP first-client range
- Docker: new espidf-builder stage with ESP-IDF 4.4.7 toolchain
- 157 tests covering WiFi/BLE for both ESP32 (Xtensa) and ESP32-C3 (RISC-V)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
David Montero
ZhadowValker