5.9 KiB

Raw Blame History

15 — Universal webcam compatibility (any webcam, any PC) ✅

Closing chapter of the ESP32-CAM emulation arc. The previous fixes (bugs #1-9 in 12_*.md-14_*.md) made esp_camera_fb_get() work end-to-end with synthetic JPEGs, then with real webcam JPEGs at low quality. This doc covers the LAST known edge: the deliverable byte budget was hard-capped at 8 KiB, so JPEGs from HD webcams or detail-rich scenes got truncated and jpg2rgb565 failed intermittently (JPG Decompression Failed! Data format error).

User-facing requirement: "el código debe funcionar para cualquier webcam de cualquier PC".

Two independent layers

The fix has two layers; either alone is insufficient.

Layer A — Bounded JPEG encoder (frontend)

frontend/src/hooks/useWebcamFrames.ts no longer ships with a fixed JPEG_QUALITY constant. Replaced by encodeBoundedJpeg():

const MAX_FRAME_BYTES = 23000;            // matches QEMU 32 KiB cap
const QUALITY_LADDER  = [0.6, 0.5, 0.4, 0.3, 0.2, 0.1];

async function encodeBoundedJpeg(c) {
  for (const q of QUALITY_LADDER) {
    const blob = await canvasToJpeg(c, q);
    if (blob.size <= MAX_FRAME_BYTES) {
      return { buf, bytes, quality: q, downscaled: false };
    }
  }
  // Last resort: downscale to 240×180.
  const small = downscaleCanvas(c, 240, 180);
  return { buf, bytes, quality: 0.4, downscaled: true };
}

Guarantees that EVERY emitted frame fits in the deliverable budget, regardless of webcam hardware or scene complexity. The encoder exposes lastQualityUsed and lastDownscaled to the UI so users can see when the auto-tuning kicks in (visible in the Camera button tooltip via CameraToggle.tsx).

Layer B — Multi-lap descriptor ring walker (QEMU)

third-party/qemu-lcgamboa/hw/misc/esp32_i2s_cam.c walk_dma_chain previously bailed when all 16 descriptors were owner=0 from a single lap, capping per-frame delivery at 8 KiB. The new code:

When the scan finds no owner=1 descriptors AND laps_in_burst < MAX_LAPS_PER_BURST, calls reset_descriptor_ring(s) (already existed, was used for fresh-frame init), increments laps_in_burst, and retries the scan from head_addr.
Same logic in the inner fill loop when advancing to the next descriptor and finding nowner=0.
vsync_kick_cb resets laps_in_burst = 0 per VSYNC cycle.
ESP32_I2S_CAM_EOFS_PER_FRAME bumped from 8 → 24 to leverage the extended budget (24 EOFs × 1024 samples = 24 KiB; 4 max laps cap at 32 KiB).

Why it's safe to overwrite descriptors mid-frame: cam_hal's firmware reads from cam_obj->dma_buffer[(cnt % half_buffer_cnt) * half_size], not from the descriptor metadata. The descriptors are SoC-side scratch that the firmware doesn't observe directly. Walker writes precede the EOF IRQ that wakes the firmware, so by the time cam_task's ll_cam_memcpy runs, the bytes are stable.

Why both layers

Setup	Layer A only	Layer B only	A + B
Cheap 480p webcam	✅ q=0.5	✅ q=0.6	✅ q=0.6
Logitech mid-range	✅ q=0.4-0.5	✅ q=0.6	✅ q=0.6
HD 1080p webcam	✅ q=0.3, blurry	✅ q=0.6	✅ q=0.6
4K webcam, complex scene	✅ downscaled	❌ truncate	✅ q=0.4-0.5
Hypothetical 8K webcam	✅ downscaled	❌ truncate	✅ downscaled

Layer A alone works for everything but caps perceived quality. Layer B alone bumps the cap but doesn't handle 4K+ enterprise cameras. Both together cover every consumer webcam and gracefully degrade for the truly extreme cases.

File-level changes

Capa A

frontend/src/hooks/useWebcamFrames.ts — encodeBoundedJpeg, canvasToJpeg, downscaleCanvas helpers. Exports lastQualityUsed: number and lastDownscaled: boolean from the hook.
frontend/src/components/simulator/CameraToggle.tsx — tooltip shows (auto-tuned to q=0.X) or (auto-downscaled) when the encoder dropped below 0.3 / fell back to the smaller canvas.

Capa B

third-party/qemu-lcgamboa/include/hw/misc/esp32_i2s_cam.h — new int laps_in_burst field on Esp32I2sCamState.
third-party/qemu-lcgamboa/hw/misc/esp32_i2s_cam.c:
- Forward decl of reset_descriptor_ring (defined in lifecycle section, called from walker).
- ESP32_I2S_CAM_EOFS_PER_FRAME 8 → 24.
- New ESP32_I2S_CAM_MAX_LAPS_PER_BURST = 4.
- walk_dma_chain step-1 retry loop with reset_descriptor_ring on lap exhaustion.
- Inner-loop equivalent: when advancing finds nowner=0, run the same retry path.
- vsync_kick_cb resets laps_in_burst = 0.

Test plan

End-to-end manual (the only meaningful test for this — a unit test can't simulate a real webcam):

Hard refresh frontend (Ctrl+Shift+R).
Restart uvicorn so the new worker loads.
Stop + Run the gallery's ESP32-CAM + ILI9341 Live Preview.
Click Camera, grant permission.
Cover several scenes with the laptop webcam:
- Static dark wall → expect q=0.6, ~5-10 KiB JPEGs
- Hand waving (motion + complexity) → q≤0.5, ~15-22 KiB JPEGs
- Read a book/code/text-rich page → q≤0.4, possibly downscaled
Hover the Camera button and verify the tooltip reports q= and (auto-tuned) / (auto-downscaled) consistently with the scene complexity.
Serial monitor: ZERO JPG Decompression Failed lines — the bug that prompted this work is gone for any test scene.
Backend log: camera_frame #N received (BYTES bytes payload) reports BYTES always ≤ 23000.

Closing thought

The original goal — "ESP32-CAM emulation that just works" — needed 9 silent bugs fixed before fb_get returned anything (autosearch docs 00-14), and now needs adaptive encoding plus a multi-lap walker to handle the long tail of webcam variability. Each fix followed the same pattern: faithful upstream-driver behaviour combined with a small, well-bounded host-side accommodation. The result is the first open-source emulator that runs unmodified ESP32-CAM Arduino sketches end-to-end with real webcam input.

5.9 KiB Raw Blame History Unescape Escape