Three coordinated fixes that together close the "ESP-IDF compile takes
5-7 min every time" gap and prevent the failure mode where a user clicking
compile multiple times spawns six ninja processes that peel each other
apart on a modest VPS.
What was wrong
- /compile/start generated a fresh uuid4 every call, so 6 clicks = 6
independent builds racing each other. Saw load average 30 on the prod
VPS during a real BMP280 attempt today.
- No concurrency limit anywhere; asyncio.create_task() fired without
gating.
- ccache was wired in last week (PR #149) but reported 18,350 cacheable
calls and **0 hits** because the build dir was a fresh
tempfile.TemporaryDirectory(prefix='espidf_') per compile. The random
/tmp/espidf_<random>/ path baked into -I and -fmacro-prefix-map flags
→ different command line every compile → ccache hash miss every time.
What this PR does
1. Job deduplication (`backend/app/api/routes/compile.py`)
- New `_job_key(files, board_fqbn)` returns SHA-256 of normalised file
names + contents + board. Order-independent.
- New `JOB_BY_KEY: dict[str, str]` indexes hash → job_id.
- `compile_start` checks JOB_BY_KEY before spawning a new task; if a
job for this exact content is already pending or running, returns
the existing job_id (logs `[compile] dedup hit — reusing job <id>`).
- `_purge_expired_jobs` evicts both COMPILE_JOBS and JOB_BY_KEY,
keeping the index consistent. Edge case where two jobs share a key
(old finished, new running) is handled — only evict the key entry
if it still points at the purged job.
2. Concurrency control (`backend/app/api/routes/compile.py`)
- `_COMPILE_SEMAPHORE = asyncio.Semaphore(2)` global cap on
simultaneous compiles.
- `_target_lock(board_fqbn)` returns a per-target asyncio.Lock so
concurrent compiles to the SAME board (sharing the persistent build
dir) serialise. Different boards still run in parallel up to the
semaphore cap.
- `_compile_job` acquires sema → per-target lock → flips state to
`running` → calls `_run_compile`. Pending state now accurately
reflects "queued waiting for resources".
3. Persistent build dir (`backend/app/services/espidf_compiler.py`)
- New `_prepare_persistent_project_dir(idf_target)` materialises
`/var/lib/velxio-build/<target>/project/` from the template on
first use; on subsequent compiles it wipes only `main/` and
`user_libs/` (the per-compile parts) and leaves `build/` alone so
ninja's incremental cache + ccache .o files survive.
- Toolchain version sentinel (`.idf_version`) wipes the whole target
dir if the ESP-IDF or arduino-esp32 version changes — cached
objects from the old toolchain are no longer ABI-compatible.
- `compile()` is now a thin dispatcher: persistent path or fallback
to the legacy `tempfile.TemporaryDirectory()` flow. The actual
build logic was extracted into `_compile_in_dir()` so both paths
share one implementation, no duplication.
- Escape hatch: `VELXIO_PERSISTENT_BUILD_DIR=0` env var falls back
to the tempfile path without rebuilding the image. Critical for
production safety.
4. ccache normalisation (`Dockerfile.standalone`)
- + `ENV CCACHE_BASEDIR=/var/lib/velxio-build` makes ccache canonicalise
absolute paths under that prefix when computing the cache key.
Robustens hits against any future subdir rearrangement.
5. Docker compose (`docker-compose.yml`)
- + named volume `velxio-build:/var/lib/velxio-build` so the persistent
build dir survives `docker compose up -d --build`.
- + env `VELXIO_PERSISTENT_BUILD_DIR=1` (default ON; users disable
without rebuilding).
Expected impact
- Cold first compile per container per target: unchanged (~5-7 min).
- Same sketch re-compiled: ~2-5 s (everything cached).
- Different sketch, same target: ~5-30 s (only user code + new lib steps
rebuild; ESP-IDF base hits cache).
- Different sketch with new libraries: ~30-90 s (new lib component
compiles; rest hits cache).
- Concurrent clicks on same example: 1 build, others poll the same
job_id. No more six-ninja meltdown.
Tests
- `test/backend/unit/test_compile_dedup.py` covers `_job_key` stability +
variance and `_purge_expired_jobs` consistency (including the
"two jobs share a key" edge case).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
c2fe1af250