EPIC-09: device-side visibility and recovery

[diegoparrilla]

Second epic of v1.1.0: the device reports its own health, and a crash or hang reboots it with the reason kept instead of freezing silently.

What changes

• Crash reboots (D-03). panic() goes to health_panic (PICO_PANIC_FUNCTION), and a HardFault handler is installed in the RAM vector table at boot. fatfs-sdk's own handler stopped at a breakpoint, which locks up an RP2040 with no debugger. Both write the reason, PC, LR and SP to watchdog scratch registers 0-3 and reboot after 100 ms.
• Watchdog (D-04). 8 s, in both build types, paused while a debugger halts the CPU. It is fed from the main loop, emul_pollTick, the Wi-Fi connect loop, the HTTP spin-waits and two slow boot steps. A phase ID says where a hang happened.
• A hang is told apart from a probe reset. A debug probe resets with SYSRESETREQ, which leaves every watchdog and reset register exactly as a real expiry does. A once-a-second timer interrupt writes a stall mark after 5 s without a feed, and only a reset with that mark counts as a hang.
• Reset reason on the menu. Row 2 shows Recovered: panic @…, Recovered: fault @… or Recovered: hang in <phase>. After 3 crash reboots within 60 s the boot countdown stays stopped until a SELECT reset or a power cycle.
• Health report without a console. GET /api/v1/system/health and sidecart.py health report heap free and minimum, sbrk and stack high-water marks, code in RAM, the last reset with its phase or PC, crash count, ROM3 overruns and debug-byte drops. Debug builds print a summary every 30 s.
• lwIP statistics opt-in. They were keyed on NDEBUG, which both build types define. A debug build with DEVOPS_LWIP_STATS=1 now adds the pool counters to the health report. Release lwIP objects are unchanged.
• ROM3 ring overruns are counted, and the DMA capture is re-armed before its transfer count runs out. The protocol parser rejects a payload size larger than its buffer, which a corrupted frame could overflow.
• fatfs-sdk pinned to 6c644cf, the commit Booster uses. After a card pull, sd_read_blocks returned without releasing its locks, so the next SD call deadlocked the RP. 6c644cf is the upstream fix plus a README change.
• sd_timeouts.c overrides fatfs-sdk's weak timeout table. Debug-only fault-injection routes under POST /api/v1/debug/test/. docs/api.md, README and CLAUDE.md updated.

Verified on hardware

On release and debug, with the ST in the setup menu, [G] and [U]:

Check	Result
Panic, HardFault	Reboot in 6 to 12 s; the reason and PC on the menu, the console and health; the PC resolves to the faulting line
Hang in the main loop, hang inside an HTTP handler	Watchdog reboot, hang in main_loop / hang in http_request
Three crashes within 60 s	Countdown stopped with x3; a SELECT tap clears it
Debug-probe reset and probe flash	reboot, crash count 0
30 s summary line, lwIP counters, ROM3 overruns under a forced stall	All present; overruns count exactly the stalls that hit a debug burst
Boot with no SD card, no reachable network, SELECT held 8 s, long debugger halt	No reboot
SD card pulled	Before the fatfs-sdk pin: deadlock and watchdog reboot. After: every request fails in under a second, no reboot
RP crash while the ST is in [G] or [U]	The ST keeps running; GEMDRIVE answers again, Runner reconnects after the countdown
cli/test_sidecart.py	115 tests pass

Build	Heap total	Heap free, idle	Code in RAM
release, before this PR	36,132	—	47,224
release	35,508	26,640	47,896
debug	27,284	18,416	51,964

Found and filed, not fixed here

• The HTTP idle sweeper closes uploads and slow downloads after a few seconds, so a 4 MB upload fails (EPIC-13).
• gemdrive mkdir panics in malloc on a 29 GB FAT32 card (EPIC-11).
• A reinserted SD card is not mounted until the RP restarts; the CLI prints a traceback when the device resets a connection (EPIC-15).
• The core-0 stack reaches about 6.2 KB against its 2 KB reservation (EPIC-12).

The 4 MB upload and 10 s runner load false-reboot checks wait for the idle-sweeper fix.