list_installed_libraries() now honors VELXIO_FALLBACK_SKETCHBOOK (pro overlay) so
GET /api/libraries/list enumerates the content-addressed cache instead of the
shared global dir — the Installed tab survives the global volume's retirement
(audit finding #3). Unset (OSS self-host) -> default sketchbook, unchanged.
The last paths that read the shared global /root/Arduino/libraries: a compile
with NO manifest (libraries=null — any from-scratch/anon sketch; the manifest is
never auto-derived from #includes) and the incomplete-manifest scan-all retry
(which re-enters compile unscoped). Both fell through to the global dir,
bypassing the cache entirely (a cached lib still failed when global was gone).
Now, when no scope is materialized, point the library search at the cache: the
cache root is itself a valid Arduino libraries dir (each <name@ver-sha> child is
a library), exposed via env (pro overlay sets them):
- arduino-cli: ARDUINO_DIRECTORIES_USER = VELXIO_FALLBACK_SKETCHBOOK (whose
libraries/ -> cache root) when scope_dir is None.
- ESP-IDF: _find_arduino_libraries_dir() prefers VELXIO_FALLBACK_LIBRARIES_DIR.
Unset (OSS self-host) -> legacy global, unchanged.
Also strip a trailing @version from manifest names (_bare_lib_names): norm_name
fused 'ArduinoJson@6.21.5' -> 'arduinojson6215' (cache miss -> global scan-all);
the per-board boards_json manifest is the path that still carries @version.
Empirically, arduino-cli's --libraries ADDS to the search path — the global
sketchbook is STILL scanned, so the prior commit did NOT isolate reads from the
shared global volume. Point ARDUINO_DIRECTORIES_USER at the scratch sketchbook
instead (its <scratch>/libraries becomes the ONLY user-library dir); cores +
board-manager URLs live in the DATA dir and are untouched.
In-container functest (AVR uno): a cache-only lib compiles via the scope (never
in global); a global-only lib NOT in the manifest is INVISIBLE to the scoped
compile and only recovers via the scan-all retry (manifest_incomplete=True) —
proving the global volume is no longer scanned for a manifest-scoped build.
arduino-cli compiles now read libraries from a per-compile --libraries dir of
symlinks materialized by the pro overlay (owner store -> cache -> legacy), via
the existing materialize_library_scope hook, instead of scanning the shared
mutable global volume. --libraries only overrides the USER library search path,
so cores + board-manager URLs (RP2040 earlephilhower, ATTinyCore) are untouched.
Mirrors the ESP-IDF P2.1e graceful fallback: an incomplete manifest (a needed or
transitive lib not declared) makes the scoped compile miss a header; we retry
ONCE scan-all (no --libraries) and surface manifest_incomplete, so a partial
manifest degrades to legacy behavior instead of hard-failing.
Dedup correctness: the manifest + owner are resolved ONCE (shared
_resolve_compile_scope) and folded into the /compile/start dedup key AND threaded
into the build, so two owners with identical sketch+board but different custom
libs never coalesce to one another's job, and the key never diverges from the
bytes the build uses. Owner folded only when a manifest applies (index-only
compiles keep cross-owner dedup).
STM32 emulation (open-core, runs via libqemu-arm in the backend worker):
- backend: stm32_lib_manager + stm32_worker (GPIO, USART, I2C/SPI device models
reusing the ESP32 slaves, live sensor updates), arduino_cli STM32 branch,
start_stm32 simulation route.
- frontend: Stm32Bridge + Stm32BluePill(/BlackPill) web components (Wokwi SVGs),
board kinds, Interconnect/boardPinMapping/boardProtocols wiring, example
projects (blink, serial, I2C BMP280/MPU6050/DS1307/SSD1306/weather, 7-seg,
RGB, button, switch, stepper, cross-board interconnect).
- Raspberry Pi 4/5 board elements + thumbnails.
Pro board gating (generic OSS->Pro seam; entitlement logic lives in the overlay):
- lib/proBoardGate.ts: isProBoardKind (STM32 + every QEMU Raspberry Pi),
installBoardGateImpl/boardGateDecision, triggerProUpgradePrompt.
- PRO badge on those boards in the component picker; gate at the picker add +
the run backstop (startBoard).
- backend/app/services/board_access.py: server-side enforcement seam for the
simulation WebSocket; STM32/Pi unavailable -> Pro-framed message.
- desktop: generic QemuDownloadPrompt + Stm32QemuPrompt (download-behind-license,
mirrors the ESP32 prompt).
- .gitignore: never ship libqemu-* binaries in the public image.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ATTinyCore >=1.5.0 declares ATTinyCore:micronucleus@2.5-azd1b as a tool
dependency, hosted at https://azduino.com/bin/micronucleus/. That host
has been unreachable (connection refused) for extended periods, causing
every ATtiny85 compile to fail at the core-install step with:
Download failed: performing HEAD request: ... dial tcp ...: connection refused
Failed to install required core: ATTinyCore:avr
micronucleus is only used for USB upload — never for compilation — but
arduino-cli refuses to install a core whose tool deps cannot fetch.
Pin to 1.4.1, the last release whose micronucleus binary is hosted on
github.com (digistump release, reachable). The FQBN clock options we
ship (clock=16pll on attinyx5, etc.) are unchanged across 1.4.x.
- backend/app/services/arduino_cli.py: new CORE_INSTALL_VERSIONS map
consulted by ensure_core_for_board so the runtime auto-install
passes "ATTinyCore:avr@1.4.1" instead of unversioned latest.
- backend/Dockerfile and docker/entrypoint.sh: same pin so a fresh
image bakes 1.4.1 in and never hits the runtime fallback path.
Existing regression tests in test/backend/unit/test_arduino_cli_attinycore.py
still pass (they assert presence, not version).
Backend:
- api/routes/compile.py accepts board-specific compile options
and dedups in-flight identical requests
- services/espidf_compiler.py expanded ESP-IDF wrapper with the new
options surface (sdkconfig.defaults.in
template added)
- services/arduino_cli.py honour the new options envelope
- services/esp32_lib_bridge.py thread board options through to QEMU
Tests:
- tests/test_compile_request_dedup.py end-to-end dedup behaviour
- tests/test_espidf_options.py covers the new options parsing
Frontend:
- services/compilation.ts client-side mirror — sends the new
options field on every compile request
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Backend:
- Add version field to InstallLibraryRequest
- Add fallback and requested_version to InstallResponse
- Add DELETE /api/libraries/uninstall endpoint
- Enhance install_library() for versioned installs (LibName@version)
- Add semver validation and fallback logic
- Add uninstall_library() method
- Fix _parse_version() to reject non-numeric version parts
Frontend:
- Update installLibrary() with optional version parameter
- Add uninstallLibrary() and resolveLibraryVersion() helpers
- Add version selector dropdown in Library Manager
- Add UNINSTALL button for installed libraries
- Show fallback messages when requested version unavailable
- Add parseLibSpec() and version badges in InstallLibrariesModal
- Implemented handshake tests to validate initial bus state and register responses.
- Created end-to-end tests for Pico W LED blinking using MicroPython firmware.
- Added SDPCM framing tests to ensure proper encoding and decoding of control frames.
- Developed IOCTL tests to verify command responses and state changes in the emulator.
- Established a full lifecycle test for WiFi operations, including scanning, connecting, and packet handling.
- Introduced TypeScript configuration for test files to ensure compatibility and strict type checking.
- Updated components-metadata.json with new generated timestamp.
- Refactored Esp32C3Simulator.ts to remove unnecessary debug variables and logging, and added support for additional ROM functions.
- Modified useSimulatorStore.ts to clarify bridge usage for ESP32 boards.
- Updated submodules for QEMU and other libraries to indicate dirty state.
- Added test_esp32c3_emulation.py for end-to-end testing of ESP32-C3 emulation, including compilation, flash image merging, and GPIO event checking.
- Added detailed logging for GPIO changes, system events, and errors in simulation websocket.
- Improved ESP32 firmware handling by merging individual binaries into a single 4MB flash image.
- Updated ESP32 bridge to handle serial output and GPIO changes with appropriate logging.
- Introduced integration test for ESP32 emulation, covering compilation, WebSocket connection, and event handling.
- Enhanced examples to include ESP32 projects and updated the examples gallery to reflect new board types.
- Refactored simulator store to manage ESP32 bridge and simulator instances more effectively.
- Updated requirements to include esptool for ESP32 firmware management.
- Added ESP32 emulation plan and architecture documentation.
- Created `esp_qemu_manager.py` for managing ESP32 QEMU instances.
- Modified backend API routes to support ESP32 firmware loading and GPIO handling.
- Introduced `Esp32Bridge.ts` for frontend communication with ESP32 instances.
- Refactored simulator store to support multiple boards, including Raspberry Pi and Arduino.
- Created `RaspberryPi3Bridge.ts` for WebSocket communication between frontend and backend for Raspberry Pi.
- Updated QEMU manager to handle multiple serial ports for Raspberry Pi GPIO communication.
- Enhanced SimulatorCanvas to render multiple boards and manage wire routing between them.
- Implemented board picker modal for selecting and adding boards to the canvas.
- Updated editor to support multiple file groups per board.
- Added migration logic for loading old project formats into the new multi-board structure.
- Ensured backward compatibility with existing components and functionality.
- Add LoginPage and RegisterPage for user authentication.
- Create UserProfilePage to display user projects.
- Implement ProjectPage for viewing and editing individual projects.
- Introduce authService for handling user login, registration, and session management.
- Add projectService for managing project data retrieval and manipulation.
- Enhance EditorPage with file management capabilities and save prompts.
- Introduce Zustand stores for managing authentication, editor state, and project state.
- Add reserved usernames utility to prevent certain usernames during registration.
- Update compilation service to handle multiple files for Arduino sketches.
- Simplified serial data handling in `useSimulatorStore` for both AVR and RP2040 simulators.
- Introduced `boardPinMapping.ts` to map wokwi-element pin names to simulator GPIO/pin numbers for Arduino Uno and Nano RP2040.
- Added `compilationLogger.ts` to parse compile results into structured log entries for better console output.
- Added SimulatorCanvas component for rendering the simulator interface.
- Integrated Wokwi components (Arduino, LED, Resistor, Pushbutton, Potentiometer) into the simulator.
- Created PinManager to handle pin state changes and notifications.
- Developed AVRSimulator class for emulating Arduino Uno functionality.
- Implemented hex file loading and compilation service.
- Added CSS styles for the simulator interface.
- Established Zustand stores for managing editor and simulator states.
- Created utility functions for parsing Intel HEX format.
- Set up Vite configuration for the frontend project.
- Added batch scripts for starting backend and frontend servers, and updating Wokwi libraries.
David Montero
ZhadowValker