The 7.5" 800x480 dashboard (GxEPD2_750_T7) rendered blank: it is a UC8179 /
GD7965 controller, but the panel config claimed controllerFamily 'ssd168x',
so the SSD168x decoder (which only reads 0x24/0x26/0x44/0x45) ignored its
0x10/0x13 DTM stream.
- Add a Uc8179 decoder (worker Uc8179EpaperSlave + browser Uc8179Decoder).
UC8179 is the same UltraChip command family as the UC8159c (0x10/0x13 DTM,
0x12 refresh) but mono (1 bit/px). GxEPD2 writes the visible image to 0x13
(DTM2 "current"; 0x10 is the ignored "previous"), framed by 0x91/0x90
(partial window, pixel coords MSB-first)/0x13 data/0x92. Data lands at
absolute pixel coords inside the window, so compose is just the RAM. The
Frame reuses the SSD168x palette (0=black, 1=white) so paintFrame renders it.
- EPaperPanels.ts: add the 'uc8179' family and point epaper-7in5-bw at it.
EPaperPart.ts + esp32_worker.py dispatch 'uc8179' to the new decoder.
- Fix the BUSY polarity: UC8179 (like the UC8159c) idles BUSY HIGH, not LOW.
The worker seeded BUSY LOW for every non-uc8159c panel, so GxEPD2_750_T7's
_PowerOn()/_InitDisplay() busy-wait timed out (~10 s, "Busy Timeout!") on
every refresh. Now _PowerOn returns in ~129 us.
- esp32_worker.py: the runtime sensor_attach epaper path still emitted the
epaper_update payload nested under 'data' (the old double-wrap bug); emit
it flat like the init path.
The 5.65" ACeP UC8159c example already rendered (it has its own decoder and
got the WS-plumbing fix); verified the 7 colour bars are correct.
ePaper panels rendered rotated/misaligned on AVR and RP2040 (e.g. the 2.13"
Pico clock came out sideways and clipped). The ESP32 worker decoder was just
taught to compose in the controller's native RAM geometry and rotate to the
display orientation, but the browser-side SSD168xDecoder (used by AVR/RP2040)
still composed at display dims with no rotation, so the two diverged.
- SSD168xDecoder.ts: port the worker's native-window compose + rotation.
* Size RAM to the longer side both ways so a rotated native layout
(128x296 behind a 296x128 panel) isn't truncated.
* Compose in the active RAM window, then rotate via the inverse of
Adafruit_GFX setRotation(1). Detect orientation by BYTE width so a
non-multiple-of-8 native width (the 2.13" panel is 122 px) is handled.
* Track the UNION of windows per frame: paged drivers (GxEPD2 page height
< panel) set one partial window per page, so compose must use the full
native area, not just the last page's strip. Fixes the all-white render
on paged panels (1.54" Uno, 4.2" Pico, 7.5" ESP32).
* Add an isBwr option: B/W panels treat 0x26 as a 2nd mono plane (white
only if both planes white), tri-colour panels keep red-wins.
* Default the active window to display geometry; the firmware overrides it.
- EPaperPart.ts: pass isBwr = cfg.palette === 'bwr' to the decoder.
- esp32_spi_slaves.py / esp32_worker.py: mirror the byte-aware rotation +
window-union in the worker, and derive is_bwr from panel_kind on the
runtime sensor_attach path too (fixes the tri-colour ESP32 alert badge).
- test_epaper/ssd168x_decoder.py: re-port the golden reference to match
(keeps the 3-way TS/Python/worker identity invariant). Tests updated to
construct tri-colour cases with is_bwr/palette='bwr'.
- examples-displays-epaper.ts: the Pico VCC wire referenced '3V3(OUT)',
which the velxio-pi-pico-w element doesn't expose (it has '3V3'), so the
wire snapped to the board corner. Use '3V3'.
ESP32 ePaper examples (e.g. epaper-2in9-esp32-weather) rendered as a blank
white panel. Two bugs, both above the SPI layer:
1. The worker's epaper_update event nested its payload under 'data', unlike
every other (flat) worker event. The backend qemu_callback re-wraps the
post-'type' payload under 'data', so the frontend received
msg.data.data.component_id (undefined) and EPaperPart bailed on
id !== componentId, so paintFrame/putImageData never ran. Emit it flat.
2. Ssd168xEpaperSlave was sized to the display dims (296x128), but GxEPD2
with setRotation(1) writes the controller's NATIVE RAM (128x296). The
_y < height bound dropped rows 128-295 (half the image) and compose
never rotated. Size RAM to the longer side, compose in the native
active-window geometry (0x44/0x45), then rotate to the display
orientation (inverse of Adafruit_GFX rotation 1). Add is_bwr (from
panel_kind): B/W panels init the 0x26 plane white and compose
white-only-if-both (GDEY029T94 mirrors the image into 0x26); tri-colour
panels keep red init 0x00 and red-wins.
Worker side of the libqemu picsimlab_spi_event_batch / CS-gating change:
- _on_spi_batch(): replay a whole SPI transfer in bulk (custom-chip runtime,
then ePaper feed, then the spi_batch buffer) instead of one _on_spi_event per
byte. Registered as a trailing _SPI_BATCH field of _CallbacksT.
- _sync_cs_events(): disable SPI chip-select callbacks for pure-display sims,
enable them when an ePaper / custom-chip SPI slave is registered (no-op on
older libqemu without qemu_picsimlab_enable_spi_cs_events).
- _on_pin_change(): flush the SPI batch before each gpio_change so the byte
stream stays ordered against the DC pin now that CS no longer triggers the
flush.
Backward compatible: an older libqemu never calls the batch callback or the CS
setter, so it just keeps the per-byte path. esp32-doom: 0.04 -> ~1.0-1.5 FPS
wall-clock (~26-37x), render verified correct.
A 1024-byte oled.show() writevto generates ~1025 calls into
_on_i2c_event, one per byte. The previous code called _log (stderr
write+flush) AND _emit (stdout JSON write+flush) for every event,
saturating the worker subprocess's stdout pipe. The QEMU thread blocks
on the synchronous write, the firmware's ESP-IDF i2c_master ISR
re-enters before the previous one finished, and the Interrupt watchdog
trips on CPU1 with a "Guru Meditation Error: Interrupt wdt timeout"
panic on the second consecutive oled.show() call.
I2CWriteSink already buffers writes internally and emits a single
i2c_transaction event on FINISH, so the per-byte log+emit was pure
overhead with no observability value for display drivers (SSD1306,
PCF8574). Keep them for everything else.
Verified end-to-end via chrome devtools MCP:
- Test minimal (init + 1 explicit show): markers OK, no panic
- Loop test (10x oled.show() + sleep(0.5)): DONE_LOOP, no panic
Two intertwined bugs were leaving every ESP32 ePaper example broken
end-to-end. Only the 5.65" UC8159c panel surfaced the failure
audibly ("Busy Timeout!" repeating in serial), because its inverted
busy polarity caused the firmware to hang inside `_waitBusy()`. The
SSD168x ePaper examples APPEARED to run cleanly but never actually
rendered anything to the panel — the canvas stayed at the idle paper
colour because the same registration path was broken.
Root cause #1 — `setSensors` was a full REPLACE, not a merge.
`Esp32Bridge.setSensors(sensors)` did `this._pendingSensors =
sensors`. At `startBoard()` time the store iterates components,
resolves wires for any entry in `SENSOR_COMPONENT_MAP` (DHT22 /
HC-SR04 / I²C sensors) and calls `setSensors(...)` with that list.
ePaper components live in `PartSimulationRegistry` (not in the
sensor map) and are registered via `sendSensorAttach()` AT
COMPONENT-MOUNT TIME — well before `startBoard()` runs. Full-replace
semantics blew that registration away on every Run click, so the
worker never instantiated an `Ssd168xEpaperSlave` / `Uc8159cEpaperSlave`,
no SPI bytes were decoded, no frames were latched, and BUSY was
never driven.
Fix: upsert by `pin` so pre-existing registrations from
PartSimulationRegistry handlers are preserved alongside the
startBoard-resolved sensors. Confirmed via a WebSocket spy that the
`start_esp32` payload now carries the ePaper sensor entry.
Root cause #2 — BUSY polarity was hard-coded for SSD168x only.
Verified against upstream GxEPD2 source:
* SSD168x family — constructor passes `_busy_level = HIGH`
→ BUSY=HIGH means busy, LOW means ready.
* UC8159c family — constructor passes `_busy_level = LOW`
→ BUSY=LOW means busy, HIGH means ready.
The worker only drove BUSY after a frame flush (and at the wrong
polarity for UC8159c), so the firmware's first `_waitBusy()` inside
`_PowerOn()` / `_InitDisplay()` — which fires BEFORE any frame —
blocked for the full 25 s `_busy_timeout`.
Fix: read `controller_family` from the registration payload, pick the
per-family idle level, and (a) seed the pin to IDLE at registration so
the first `_waitBusy()` sees "ready" immediately, (b) use that
polarity (idle vs. busy) when pulsing on frame flush.
Verified on https://velxio.dev/example/epaper-5in65-7c-esp32-rainbow:
the serial timeline now reads `_InitDisplay reset : 1566` /
`_PowerOn : 148` / `_PowerOff : 183` / `frame done` (all sub-2 ms
busy-waits, no timeouts). Sensor registration confirmed via the
`start_esp32` payload carrying the `epaper-ssd168x` entry.
Two CI failures landed after PR #196 (esp32-gpio-matrix-cb-callback)
merged. Both are independent and fixed here together.
1) **Backend E2E: ESP32 hangs at bootloader handoff.**
PR #196 added picsimlab_gpio_matrix_cb which fires on QEMU's
iothread. The handler did `_emit({...})` for every routing
change — and the ESP-IDF bootloader writes to gpio_out_sel
*hundreds* of times during early boot (each peripheral init
configures its matrix slot). Each emit acquires _stdout_lock
and writes to the worker→manager pipe. If the manager drains
even briefly slow, the pipe fills, write blocks, and the
iothread stalls — symptom: ESP32 reports `entry 0x400805e4`
then no Arduino setup() output for 75 s.
Fix: the iothread callback now ONLY mutates the SignalRouter
snapshot. It never emits. The 10 Hz poll thread
(_refresh_signal_routing) stays as the sole emitter, so the
wire-format event stream is unchanged. Benefit of having the
callback over poll-only is reduced worst-case routing-emit
latency (next poll tick vs up to 100 ms) and a warmer
snapshot dict for cheaper poll diffs.
2) **Frontend Tests: Node OOM at end of suite.**
117 test files run in one forks-pool worker. Several lazy-load
the ngspice emscripten module (~30 MB), the MixedModeScheduler
singleton, and other heavy modules whose dispose hooks aren't
reached because singletons leak across files. Cumulative heap
pressure exceeds Node's 4 GB default; the worker hits "Ineffective
mark-compacts near heap limit" AFTER all 1881 tests pass and
the OOM kill is reported by vitest as "Worker exited unexpectedly
/ Timeout terminating forks worker". This is not a real test
failure — every individual test passes.
Quick fix: pass NODE_OPTIONS=--max-old-space-size=8192 to the
`npm test` step. Long-term, the singletons should add dispose
hooks that test fixtures call in afterAll(), or the suite
should shard into multiple `vitest run --shard` invocations.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires the new synchronous GPIO Matrix callback exposed by
libqemu-{xtensa,riscv32} 1.1.0 (lcgamboa/qemu commit e178ff5).
Whenever the firmware writes GPIO_FUNCx_OUT_SEL_CFG_REG, the C
plugin now fires picsimlab_gpio_matrix_cb(gpio, signal_id) inline.
The handler:
- Treats signal_id == 0x100 or 0 as "matrix routing cleared" and
emits gpio_routing_clear.
- For LEDC HS/LS range signals (the only ones the frontend
SignalRouter currently consumes), updates the mirror and emits
gpio_routing.
- Drops other signals — the mirror does not need to track them
yet, and emitting them would only fatten WS frames.
Backwards compat:
- Older libqemu (<1.1.0) doesn't expose the new field; the
picsimlab_gpio_matrix_cb placeholder runs (no-op) and the
100 ms _refresh_signal_routing() poll thread continues to feed
the mirror. WS event shape is identical either way.
Burn-in: keeping the poll thread active in parallel with the
callback for now. Once telemetry confirms parity (per phase 4 doc
in velxio-prod/project/esp32-gpio-matrix-cb/), the poll thread
gets retired in a follow-up commit.
The SignalRouter path has been in prod through Phase 2.5 / Phase 3.3
deploys without regressions, so the temporary fallback shipped in
commit 77bf897 can come out. Closes#101.
Backend (esp32_worker.py + esp32_lib_manager.py):
- Stop emitting `ledc_update` from the 0x5000 LEDC callback and from
the polling thread. Only `ledc_duty` (channel + duty_pct) and the
GPIO matrix routing events ship now.
- Drop the channel→gpio reverse-lookup that fed the legacy event.
Frontend:
- Delete `PinManager.broadcastPwm` and `PinManager.pwmListenerPinCount`.
- Delete `makeLedcUpdateHandler` + its `channelGpioMemo`.
- Delete `Esp32Bridge.onLedcUpdate` field + the `case 'ledc_update':`
message handler + the `LedcUpdate` type.
- Strip `this.onLedcUpdate = null` from 14 test mocks.
- Rewrite the `does not call broadcastPwm` guard in
esp32-multi-servo-gpio-matrix.test.ts to assert the method itself
no longer exists on PinManager (stronger regression guard than the
spy version, and doesn't need vi).
- Remove the `PinManager.broadcastPwm fallback` describe block from
esp32-servo-pot.test.ts — every test in it exercised the deleted
fallback path.
Docs (ESP32_EMULATION.md):
- Replace `ledc_update` rows in the events / implementation tables
with the SignalRouter trio (`ledc_duty`, `gpio_routing`,
`gpio_routing_clear`).
- Update the visual flow diagram + the "why this matters" paragraph
to past-tense the broadcastPwm bug.
Tests: 1886 frontend tests pass (the previously-failing
board-kinds-coverage test that needed the new Pi Zero/1/2 kinds is
also green). Backend unit suite: 279 pass, the 11 espidf_real_paths
prereq failures are environment-dependent (need arduino-cli libs in
the local shell) and unrelated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The esp32_worker.py subprocess is launched via `python <abs_path>`
and runs with a sys.path that does NOT include the backend/ package
root, so `from app.services.signal_router import SignalRouter`
raised ModuleNotFoundError at worker startup. The worker exited
with code 1 before QEMU even loaded, and the frontend surfaced the
generic "ESP32 crash detected — cache error" banner.
Mirror the existing esp32_flash_image fallback pattern (already in
this same file): try the package import first, fall back to
importlib.spec_from_file_location with the sibling .py path, then
publish the resulting module under its bare name in sys.modules so
typing references continue to work.
Verified: a synthetic test that strips backend/ from sys.path can
still construct a SignalRouter via the fallback. 20 unit tests in
test_signal_router.py still pass.
Replaces the per-peripheral ad-hoc `_ledc_gpio_map` cache with a
proper signal-routing abstraction that mirrors the ESP32 SoC's
IO_MUX + GPIO Matrix exactly. Same idea as real silicon: signal
sources (LEDC channels, RMT, MCPWM, ...) → 40-entry routing table
→ GPIO pins.
Motivation (from user bug report in
velxio.dev/project/5218f9e3-136d-43b3-bba1-6cebde21e1a4): two
ESP32 servos on a solar-tracker visibly oscillated between two
positions instead of moving smoothly when the user changed LDR
sliders. Commit 77bf897 patched it (per-channel gpio memo +
broadcast guard) but the user requested a proper hardware-fidel
architecture, not patches.
Backend:
* `app/services/signal_router.py` — SignalRouter class. Forward
index (gpio → signal_id) + reverse index (signal_id → set of
gpios). `replace_snapshot()` returns the diff for the polling-
fallback path; future C plugin hook becomes a push without
touching this code.
* `app/services/esp32_signals.py` — Signal id constants from
ESP32 TRM (LEDC HS 72-79, LS 80-87) + `ledc_signal_for_channel()`
helper.
* `app/services/esp32_worker.py` — `_ledc_gpio_map` is gone;
`_refresh_ledc_gpio_map` replaced by `_refresh_signal_routing`
which emits `gpio_routing {gpio, signal_id}` events on diff.
The 0x5000 LEDC callback and the LEDC poll thread now emit
`ledc_duty {channel, duty_pct}` (canonical, no gpio) alongside
the legacy `ledc_update {channel, duty, gpio}` for back-compat
during rollout.
Frontend:
* `simulation/SignalRouter.ts` — 1-to-1 TS mirror of the Python
class. Same forward + reverse index; same `pinsForSignal` /
`updateRouting` / `clearRouting` API.
* `simulation/esp32-signals.ts` — Signal id constants, mirror
of the Python module.
* `simulation/Esp32Bridge.ts` — new `onLedcDuty`, `onGpioRouting`,
`onGpioRoutingClear` callbacks; handlers for the new event types.
* `store/useSimulatorStore.ts` — `makeLedcDutyHandler` looks up
pins via `router.pinsForSignal(ledcSignalForChannel(channel))`
and dispatches per pin. `makeGpioRoutingHandler` /
`makeGpioRoutingClearHandler` keep the mirror in sync. Per-board
`signalRouterMap` parallels `pinManagerMap` in lifecycle.
`makeLedcUpdateHandler` (and its memo workaround from 77bf897)
stays wired for back-compat during rollout; removed in a
follow-up commit once prod is verified stable on the new path.
Tests:
* `test/backend/unit/test_signal_router.py` (20 tests) covers
update/clear semantics, idempotency, multi-pin routing,
snapshot diff, channel↔signal-id helpers, and the multi-servo
regression scenario.
* `frontend/src/__tests__/SignalRouter.test.ts` (17 tests) is the
mirror — same scenarios on the TS side.
* `frontend/src/__tests__/esp32-multi-servo-gpio-matrix.test.ts`
(6 tests) drives the end-to-end SignalRouter handler pipeline,
asserts that two servos on GPIO 13/12 via LEDC channels 0/1
move independently (no mirroring), that re-routing carries
cleanly, and — critically — that `PinManager.broadcastPwm` is
never called.
Totals: +700 LOC, 1876 frontend tests pass (was 1853), 278 backend
unit tests pass (was 259).
Docs: ESP32_EMULATION.md §9.2 rewritten with the new architecture
diagram + a runbook for adding future peripherals through the
SignalRouter.
The C plugin hook in qemu-lcgamboa that would push gpio_out_sel
writes synchronously (eliminating the polling race window entirely)
is the next step — kept as a follow-up because the polling-fallback
path here already resolves the routing before each duty event
fires, so the bug is fixed end-to-end. The plugin work removes the
race condition fundamentally.
Closes the remaining gaps in cross-board I2C so any topology of
supported boards (Uno↔ESP32, two ESP32s, Uno↔Uno↔Uno, ESP32-C3
connected to anything, etc.) works end-to-end with all I2C
components including write-only sinks (SSD1306, PCF8574, LCD-I2C).
Implementation (6 phases):
1. **BFS routing in I2CBusManager**: connectToSlave + handleExternalConnect
walk the bridge graph with a visited Set so multi-hop chains
(A↔B↔C with the device on C) resolve transparently. A new
forwarder-device shim is installed at intermediate hops so the
existing handleExternalWrite/Read/Stop machinery routes
through without per-method visited tracking.
2. **Per-peer proxy ownership in Esp32BridgeShim**: replaces the
global _proxiedAddrs Set with _proxiedByPeer Map so concurrent
bridges to the same ESP32 (e.g. wired to both Uno and Pico)
don't wipe each other's proxies on teardown. Interconnect's
per-wire teardown calls clearProxiesForPeer(peerBus) instead of
clearAllProxies.
3. **BFS-aware proxy sync**: syncProxyFromPeer now walks the peer
bus + its transitive bridges, so an ESP32 sees devices on
boards two or more hops away. _peerDeviceLookup keeps a flat
addr → device map for write-forwarding and resync.
4. **Periodic resync (250 ms)**: Esp32BridgeShim runs a setInterval
while any proxy is live, re-dumping each device with
dumpRegisters() and pushing updateProxyI2c only when an XOR-
stride hash changes. This keeps RTC time advancing visible to
ESP32 firmware without flooding the WS pipe with static
calibration dumps. Hash is primed during initial sync so the
first tick doesn't push a redundant identical buffer.
5. **Write-forwarding ProxySlave → peer**: backend ProxySlave
buffers write bytes during the transaction and emits a
`proxy_i2c_complete` event on STOP / repeated-START. Frontend
Esp32Bridge dispatches the event to a new onProxyI2cComplete
callback; the shim replays the byte sequence on the actual
peer I2CDevice via writeByte() + stop(). Makes ESP32 firmware
writes to peer SSD1306 actually repaint the OLED, peer PCF8574
latch updates, peer I2CMemoryDevice register mutations propagate.
6. **ESP32-C3 routed as bridge**: Interconnect.isBrowserSim no
longer claims c3/xiao-c3/c3-supermini — they were already
going through Esp32Bridge per the store's ESP32_RISCV_KINDS
routing, but Interconnect was treating them as browser sims
which broke proxy install. isEsp32Bridge now correctly
includes c3 family + ESP32-S3 + Arduino Nano ESP32.
Defensive: addBoard now disposes any existing shim's proxies
before overwriting simulatorMap entry so test reruns don't leak
timers.
Tests:
- 4 BFS multi-hop tests (i2c-multi-board-slave-gap.test.ts)
- 11 cross-board scenarios + per-peer + write-forward + resync
(i2c-esp32-multiboard-bridge.test.ts)
- 1 real-firmware E2E for write-forward via QEMU (compile +
load + observe proxy_i2c_complete arriving with the byte)
- New sketch fixture: esp32_i2c_write_to_peer.ino
Result: 90 test files / 1295 tests pass / 0 fail.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Implemented `i2c-esp32-real-firmware.test.ts` to test ESP32 I2C communication via backend and WebSocket.
- Created `load-example-transitions.test.ts` to ensure proper loading of examples between board-less and board-based contexts.
- Added `CircuitVerificationModal.tsx` to display circuit verification results before running simulations.
- Developed `circuitVerifier.ts` to perform pre-flight checks for circuit safety, identifying potential issues like short circuits and component overloads.
- Introduced minimal ESP32 I2C master sketch `esp32_i2c_writer.ino` for testing I2C transactions.
CI's e2e test_hcsr04_simulation.mjs caught the regression introduced by
a3f21a2 (the issue #101 fix). The worker crashes on boot with
Firmware decode error: No module named 'app'
esp32_worker.py runs as a subprocess via subprocess.Popen([sys.executable,
WORKER_PATH, ...]). When Python launches a script directly, sys.path[0]
is the SCRIPT's directory (backend/app/services/), not the backend root.
So `from app.services.esp32_flash_image import pad_to_flash_size` fails
because there is no `app/` under `backend/app/services/`.
esp32_lib_bridge.py wasn't affected because it runs in-process inside
uvicorn, where backend/ is implicitly on sys.path.
Fix: same try/except + importlib fallback the worker already uses for
esp32_i2c_slaves at the top of the file. First try the package import
(works when imported by the bridge's tests or anything else with the
backend root on sys.path), fall back to direct file loading otherwise.
Verified the fallback works in isolation by simulating the subprocess
context (sys.path containing only backend/app/services/) — the package
import fails as expected and the file-load fallback returns a properly
padded 4 MB buffer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Issue #101 reproducer: an ESP32 sketch that pulls in Adafruit_SSD1306 +
Adafruit_GFX produced a "No response from server. Is the backend
running on port 8001?" error in the browser. The compile actually
succeeded backend-side, but the JSON response carrying the firmware
was ~5.5 MB of base64 — the ESP-IDF compiler builds a full 4 MB merged
flash image (mostly 0xFF padding), encodes it whole, and ships it. In
prod that response goes through nginx + Cloudflare, which buffer-fail
or RST the connection on payloads that big — axios then lands in the
"no response" branch with no HTTP status to surface.
Fix: trim the trailing 0xFF padding before serializing, re-pad to a
valid QEMU flash size (2/4/8/16 MB) just before mtd attach. Lossless:
bytes after `last_used` in the merge are 0xFF by construction, so
trim → pad reproduces the original image byte-for-byte.
Numbers from the reproducer (Adafruit_SSD1306 + Adafruit_GFX,
esp32:esp32:esp32 board):
before: ~5.5 MB JSON response
after: 539 KB JSON response (10× smaller)
backend/app/services/espidf_compiler.py
_merge_flash_image now tracks `last_used` across the three placed
sections (bootloader / partitions / app) and writes only
flash[:last_used] to merged_flash.bin.
backend/app/services/esp32_flash_image.py (new)
Shared `pad_to_flash_size(bytes) -> bytes` helper. Rounds up to the
next valid QEMU flash size with a 4 MB minimum, matches the
frontend's existing padToFlashSize logic in Esp32MicroPythonLoader.
Raises ValueError on >16 MB inputs (would indicate a broken upstream
merge, not anything user-recoverable).
backend/app/services/esp32_lib_bridge.py
backend/app/services/esp32_worker.py
Both QEMU consumer paths (in-process and subprocess) call
pad_to_flash_size right after base64.b64decode, before writing the
tmp .bin that QEMU attaches with `-drive if=mtd,format=raw`.
Verified: smoke test confirms trim → pad → original is byte-exact.
Edge cases covered: small payloads pad up to the 4 MB minimum;
firmwares >16 MB are rejected loudly.
Closes#101
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User reported only 2 frames rendering after the SPI batching change.
Diagnosis: the previous flush triggers (CS-line HIGH, buffer >=4096)
were both event-driven. Adafruit_ILI9341's ESP32 backend manages CS
via digitalWrite — i.e. through the GPIO peripheral, NOT the SPI
peripheral's hardware CS pin. So the CS-line-HIGH event from
picsimlab_spi_event NEVER fires for this driver. The buffer only
flushes when it hits 4096 bytes.
Frame 1: 38 400 bytes from drawRGBBitmap → 9 flushes at 4096-byte
boundaries → last 192 bytes stay in the buffer. Status bar adds
some bytes too → maybe one more flush.
Frame 2: same. But by frame 3 the firmware is running ahead of the
flush rhythm and somehow the buffer pattern wedges in a state where
no flush completes (likely a partial buffer that sits between
transactions while the firmware briefly waits on the next fb_get).
Hard to reproduce deterministically — but the symptom matches.
Fix: add a 50 ms periodic flush thread. Independent of any event,
it acquires the lock and flushes whatever's pending. Bounds the
worst-case latency at 50 ms (= 20 fps ceiling, more than enough for
the emulator).
Triple-trigger now:
1. CS HIGH (fast path for hardware-CS drivers)
2. Buffer >= 4096 (safety for big transactions)
3. 50 ms timer (catches GPIO-CS drivers, prevents stalls)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User reported the ESP32-CAM + ILI9341 live preview at ~1 frame/min.
Profile: 80×60 preview pushes 9600 SPI bytes per drawRGBBitmap, and
each byte was emitting a full {type:'spi_event'} JSON message over
the worker→backend→WS→frontend pipeline. Per-byte overhead ~150-200µs
in Python (json.dumps + sys.stdout.write+flush dominates) plus
asyncio + WS dispatch. Net: 1.5-2 sec/frame minimum, much worse with
GIL contention.
Fix: buffer MOSI bytes in the worker and emit a single base64-encoded
`spi_batch` message when CS goes HIGH (transaction ended) or the
buffer crosses 4 KiB. ~9600 events/frame collapse to ~3 messages.
backend/app/services/esp32_worker.py:_on_spi_event
- Add _spi_byte_buf bytearray + threading.Lock
- On op==0x00 (byte): append; flush early if buf >= 4096
- On op==0x01 (CS change): flush buffer, then emit the CS event
via the legacy spi_event channel (ePaper / custom chips that
observe CS still get it).
frontend/src/simulation/Esp32Bridge.ts
- New 'spi_batch' message handler decodes b64 and replays each
byte through the existing onSpiByte callback. Parts that
subscribed via simulator.spi.onByte don't notice the protocol
change. The 'spi_event' branch still handles CS changes plus
legacy single-byte payloads for backwards compat.
Now that 38 KB/frame is cheap, restore preview to 160×120 + JPEG
quality 0.35 in the gallery example. Real measured speedup: ~50× on
the QVGA preview demo. Real hardware was never affected — it runs
SPI at 80 MHz and pushes the bitmap in ~4 ms either way.
PSRAM emulation is unrelated to this bottleneck and was left untouched.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
User confirmed the FULL pipeline works with their laptop webcam at
QVGA quality 0.6 — frontend → WS → backend → worker → DLL → I²S →
firmware → esp_camera_fb_get() → user sketch.
[Frame #1] 8192 bytes 320x240 fmt=4
├─ SOI (FF D8 FF): ✓ at offset 0
├─ EOI (FF D9): ✓ at offset 8190
└─ First 16 bytes: FF D8 FF E0 00 10 4A 46 49 46 …
(JFIF, real webcam)
Stats: 10/10 Valid JPEGs at ~3.5 fps.
Changes:
- wokwi-libs/qemu-lcgamboa @ e4321d1 (picsimlab-esp32):
EOFS_PER_FRAME 6→8, inject_eoi_now flag for EOI injection on the
last EOF of each VSYNC burst. Handles JPEGs of arbitrary size by
forcing FF D9 at offset 8190 — JPEG decoders tolerate the
truncation gracefully.
- backend/esp32_worker.py: throttled trace log every 30 frames
(`camera_frame #N received (NNNN bytes)`) so users can confirm
the frontend → worker leg is alive without flooding the log.
- test/test-esp32-cam/autosearch/14: documented bug #9 (the 9th and
final silent bug — real webcam JPEGs exceed the deliverable byte
budget) with full forensic trace + final architecture diagram.
The emulation now handles ANY user webcam → ESP32-CAM use case end
to end. Standard upstream esp_camera_init() / esp_camera_fb_get()
sketches work without modification.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
First open-source end-to-end emulation of the AI-Thinker ESP32-CAM
in QEMU, paired with a browser webcam → firmware bridge so users can
develop camera sketches without hardware. Status: esp_camera_init()
returns ESP_OK; OV2640 chip-id verifies (PID/VER/MIDH/MIDL exactly
match the datasheet); GPIO 25 VSYNC NEGEDGE interrupt enabled by
the upstream driver. Final piece (cam_task accepting frames) is in
progress — descriptor walker fix landed in this commit.
Backend (Python/FastAPI):
- simulation.py: camera_attach/frame/detach WS handlers
- esp32_worker.py: ctypes binding to velxio_push_camera_frame +
feature-detection fallback for older DLLs
- esp32_lib_manager.py: forward camera commands to the worker stdin
- esp-idf-template/main/CMakeLists.txt: esp32-camera headers added
via add_prebuilt_library + REQUIRES driver (resolves i2c_master_*
symbols). LED_BUILTIN=2 fallback for sketches that hardcode it.
Frontend (React/TS):
- EditorToolbar.tsx: ESP32-CAM (and the rest of the ESP32 family)
added to isQemuBoard list — Run button now starts the QEMU bridge
for these boards instead of falling through to the AVR path
- useWebcamFrames.ts: getUserMedia → OffscreenCanvas →
toBlob('image/jpeg') → base64 → WS at ~10 fps
- CameraToggle.tsx: header button with status colors + frame counter
- SimulatorCanvas.tsx: render CameraToggle for esp32-cam boards
- Esp32Bridge.ts: sendCameraAttach/Frame/Detach + chunked btoa
- useSimulatorStore.ts: diagnostic log on compileBoardProgram
- components-metadata.json: regen including esp32-cam component
Submodule pointer:
- wokwi-libs/qemu-lcgamboa → ff8eee0 (camera devices commit on
davidmonterocrespo24/qemu-lcgamboa branch picsimlab-esp32)
Investigation + tests in test/test-esp32-cam/:
- 13 autosearch markdown docs (overview, SOTA, OV2640 spec, DVP/I2S
spec, build blueprint, blockers resolved, descriptor walker fix)
- 5 sketches (camera_init, sccb_probe, dma_smoke, frame_roundtrip,
webcam_demo) + 8 live + WS regression tests
- README with the user-facing flow
.gitignore:
- libqemu-*.dll.{pre-camera,new,bak} (rollback points, regenerated)
- wokwi-libs/esp32-camera/ (clone consumed by arduino-esp32 path,
not part of this repo)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Implement UC8159cDecoder for handling 7-colour ACeP panels.
- Introduce painting functions for UC8159c frames in EPaperPart.
- Update EPaperPart to handle both SSD168x and UC8159c frame types.
- Add integration tests for EPaperPart and UC8159cDecoder.
- Create example sketch for 5.65" ACeP 7-colour panel.
- Enhance error handling in test cases for library dependencies.
- Introduced EPaperPanels.ts to define configurations for various ePaper panels including dimensions, refresh rates, and controller details.
- Implemented SSD168xDecoder.ts to handle the decoding of SPI commands for the SSD168x family of ePaper displays.
- Created EPaperPart.ts to manage the simulation of ePaper panels, integrating with the existing simulator architecture and handling events.
- Added example sketches for 2.13", 2.9", 4.2", and 7.5" ePaper displays, demonstrating basic functionality and text rendering.
- Ensured compatibility with AVR, RP2040, and ESP32 platforms, with appropriate pin configurations for each.
- Implemented `esp32_spi_chip_demo.ino` to demonstrate SPI communication with a 74HC595 shift register.
- Created `esp32_uart_chip_demo.ino` for UART loopback testing with ROT13 transformation.
- Added Python tests for compiling chips and sketches, ensuring valid WASM output and successful compilation for various board families.
- Developed end-to-end tests for ESP32 with custom chips using I2C and SPI, validating synchronous communication through the backend.
- Introduced GPIO bridge tests to verify serial communication and GPIO state changes.
- Ensured all tests validate the expected behavior of the custom chips and their interaction with the ESP32 firmware.
- Implement `ammeter-waveform.test.ts` to validate AC readings from a sine wave source.
- Create `capacitor-charge-transient.test.ts` to test the charging response of an RC circuit driven by a microcontroller pin.
- Introduce `esp32-rectifier-integration.test.ts` for testing rectifier behavior using QEMU and ESP32.
- Add helper functions in `esp32RectifierE2E.ts` for the rectifier test harness.
- Develop `voltmeter-waveform.test.ts` to ensure correct AC and DC readings from a sine wave source.
- Implement unit tests for waveform statistics in `waveform-stats.test.ts` to validate RMS, mean, peak, and interpolation functions.
- Create `waveformStats.ts` to provide statistical functions for time-domain waveform analysis.
This commit introduces a detailed markdown document outlining the process of simulating DHT22 and HC-SR04 sensors on the ESP32 platform using Velxio's QEMU fork. The documentation covers the context of the simulation, key callbacks, problems encountered, and solutions implemented for both sensors. It includes architectural details, end-to-end testing procedures, and guidelines for adding new GPIO-timed sensors. The aim is to provide maintainers with a thorough understanding of the GPIO logic and the challenges faced during development.
- Introduced I2C_SENSOR_MAP for pre-registering I2C sensors in the simulator store.
- Implemented I2C slave state machines for MPU6050, BMP280, DS1307, and DS3231 sensors in esp32_i2c_slaves.py.
- Added unit tests for I2C slave functionality covering BMP280, DS1307, DS3231, and I2CWriteSink.
- Updated the simulator store to handle I2C address resolution and sensor data management.
- Marked submodules as dirty in wokwi-libs for rp2040js and wokwi-elements.
- Implemented _MPU6050Slave and _BMP280Slave classes for I2C communication.
- Enhanced main function to register these sensors and handle I2C events.
- Updated sensor management to support MPU-6050, BMP280, DS1307, DS3231, SSD1306, and PCF8574.
- Added frontend examples for BMP280 weather station and SSD1306 OLED display.
- Modified Esp32Bridge to handle new I2C transaction events.
- Updated ProtocolParts to support ESP32 path for I2C devices.
- Enhanced useSimulatorStore to manage I2C transaction listeners.
Two fixes for ESP32 WiFi not connecting in production:
1. espidf_compiler.py: Change WiFi normalization from 'Velxio-GUEST' on
channel 6 to 'Espressif' on channel 5. The lcgamboa QEMU binary
downloaded from GitHub Releases only contains the original three APs:
PICSimLabWifi (ch1), Espressif (ch5), MasseyWifi (ch10). Channel 6
had no matching AP, so the beacon timer's channel-match condition never
fired → firmware scanned forever and never connected.
2. esp32_worker.py: Redirect fd 1 to /dev/null before loading QEMU so
raw UART bytes from QEMU's -nographic mux don't corrupt the JSON
event pipe. The real pipe fd is saved and sys.stdout is rebound so
_emit() continues to work. This also prevents stdout pipe back-pressure
from stalling qemu_main_loop() (and thus REALTIME timers).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replace arduino-cli with ESP-IDF 4.4.7 for ESP32 compilation — Arduino-compiled
firmware crashes in QEMU (9-28 reboots) while ESP-IDF boots cleanly (0 reboots).
The new espidf_compiler translates Arduino WiFi/WebServer sketches to native
ESP-IDF C code, compiles with cmake+ninja, and merges into 4MB flash images.
Key changes:
- ESP-IDF compiler: translates WiFi.begin/WebServer to esp_wifi/esp_http_server
- ESP-IDF project template with QEMU-optimized sdkconfig (DIO, 40MHz, no WDT)
- WiFi status parser for ESP-IDF serial logs (wifi_status, ble_status events)
- IoT Gateway HTTP reverse proxy for ESP32 web servers
- WiFi/BLE auto-detection from sketch content + visual status icons
- Static IP 192.168.4.15 matching slirp DHCP first-client range
- Docker: new espidf-builder stage with ESP-IDF 4.4.7 toolchain
- 157 tests covering WiFi/BLE for both ESP32 (Xtensa) and ESP32-C3 (RISC-V)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Added board-agnostic sensor registration methods in RP2040Simulator.
- Enhanced ComplexParts to handle LEDC PWM duty updates for ESP32.
- Updated ProtocolParts to check if the simulator handles sensor protocols natively, delegating to backend if applicable.
- Introduced pre-registration of sensors in useSimulatorStore for ESP32 to prevent race conditions.
- Added tests for ESP32 DHT22 sensor registration flow, ensuring proper delegation and fallback mechanisms.
- Created tests for ESP32 Servo and Potentiometer interactions, verifying PWM subscriptions and ADC handling.
- Updated components-metadata.json with new generated timestamp.
- Refactored Esp32C3Simulator.ts to remove unnecessary debug variables and logging, and added support for additional ROM functions.
- Modified useSimulatorStore.ts to clarify bridge usage for ESP32 boards.
- Updated submodules for QEMU and other libraries to indicate dirty state.
- Added test_esp32c3_emulation.py for end-to-end testing of ESP32-C3 emulation, including compilation, flash image merging, and GPIO event checking.
- Updated wire structure to replace control points with waypoints for better handling of wire paths.
- Introduced new utility functions for wire hit detection and rendering segments.
- Enhanced wire creation process to support dynamic waypoints and color assignment.
- Implemented a new ESP32 worker for improved simulation handling.
- Added utility functions for generating orthogonal paths and auto-coloring wires based on pin names.
- Improved compatibility with existing projects by ensuring backward compatibility with wire data structures.
David Montero Crespo