Repository navigation
SD card and CLI rough edges (EPIC-15) - #16
Merged
Merged
Conversation
The card was mounted once at boot, so pulling it and putting it back left GEMDRIVE and the file API dead until someone reset the device. Two separate things kept it that way. The driver never learns the card left. sd_card_spi_init() returns immediately unless STA_NOINIT is set, and with card-detect disabled on this board nothing ever sets it when a card is pulled -- so on reinsertion the driver still believed the old card was initialised, skipped its whole init sequence, and handed FatFs a freshly powered card it had never spoken to. sd_card_p->deinit() sets the bit back. And while the volume stays registered, mount_volume() never calls disk_initialize() at all, so f_mount(NULL, "0:", 0) has to come first. Both are needed: the first version of this fix had only the unregister, and every retry still failed with FR_DISK_ERR until the deinit was added. Detecting the removal cannot be left to FatFs results either. With the card physically out, volume answered 200 from the cached free space and a listing answered 200 with an empty directory -- FatFs serves what it cached and reports no error, so no failure ever arrives. The main loop now asks the card directly every 2 s by reading sector 0 through disk_read(), which the cache cannot answer. FatFs medium errors still mark the card gone, from the HTTP path and all 15 GEMDRIVE call sites, but the probe is what catches a pull. Verified with the card pulled and reinserted: detected in about 2 s, remounted about 4 s after reinsertion, volume and listings correct again, a 256 KB upload round-tripping identical, no reset and crash count 0. A device booted with no card also retries every 2 s and picks one up when it appears. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016emgLTZm7w3qZsRbukWwE4
…15 STORY-02) With no card the device was silently broken: the setup menu said nothing, [G] and [U] launched a mode with no drive behind it, and the API answered six different ways -- including two that lied. /volume answered 200 with the cached size and free space, and a listing answered 200 with an empty directory, because FatFs serves what it cached. Every endpoint that needs the card now answers 503 no_sd_card: volume, listings, downloads, uploads, folder operations, runner load and runner run, through one guard in route() and one write_no_card(). write_fs_error() reports a missing card as such rather than as a disk fault. The CLI needed no change, since it maps by HTTP status and 503 already exits 6. The menu header carries the state -- GEMDRIVE SD: mounted / SD: NO CARD -- and emul_canLaunch() refuses [G] and [U] while no card is mounted, with "Insert a working microSD card: GEMDRIVE needs one." on the status line. STORY-01's retry loop is what clears it: inserting a card lifts the block within about two seconds, with no reset. The state went on the header line rather than a row of its own because a new row breaks the menu: dividers are drawn at fixed pixel rows and the status icons are right-aligned per header row, so one extra line struck Screenmem and Hook through, collided IP address with the USB CDC heading, and ran the status line into Select an option. The framebuffer snapshot caught it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016emgLTZm7w3qZsRbukWwE4
The device dropping a connection is routine -- a watchdog reboot, a panic or a SELECT press all cut the socket -- and the user got a Python traceback ending in ConnectionResetError. urllib wraps some connection failures in URLError, which the CLI already handled in six places, and lets the rest through as bare OSError subclasses. Two exposures: urlopen() itself, and the reads after a successful open -- the download loop and the debug tail loop both call resp.read() in a loop with no handler at all, which is exactly where a reboot mid-transfer lands. CONNECTION_DROPPED names the family once, including IncompleteRead, which a chunked body cut short raises instead of an OSError. request_json re-raises it as URLError so the six existing handlers keep working and each still names its own operation; the two streaming loops report through _dropped(). A dropped download leaves the partial file in place, which is what --resume is for. DroppedConnectionTests adds a socket server that resets with SO_LINGER 0, so the client sees a real ECONNRESET rather than a clean close, and covers a reset before any reply, on a JSON command, mid-download and on health. Each asserts exit 8, no traceback, and exactly one error: line. Run against the unfixed CLI all four error with the traceback the story was filed for, so they fail for the right reason. Suite: 119 tests, all passing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016emgLTZm7w3qZsRbukWwE4
After a timeout the ST re-sends the same 1 KB chunk with a fresh token, and the RP treated it as new data and appended it again: the file gained a duplicated chunk and lost its tail. Neither side carried anything that could tell the two apart -- a retried chunk and a new one are identical on the wire. An absolute offset does not work without more state than it appears: the m68k knows only how far it has got within the current Fwrite call, not the file position, which the RP owns and Fseek moves. A relative offset cannot distinguish a retry of a call's first chunk from the first chunk of the next call, since both are zero. So the m68k carries a number that never repeats. write_seq is a longword in the GEMDRIVE blob, bumped once per chunk and never per retry, never reset, sent in d5 -- which send_sync_write_command_to_sidecart always transmits and the RP was skipping. Each open file slot on the RP remembers the last sequence it accepted and the bytes it wrote; a chunk whose sequence matches is answered with that same count and not written again. The memo is stored before the answer is sent, because the answer is what goes missing. Verified on hardware with a debug hook that stalls the answer after the data is committed. Copying a 512 KB file with two chunks stalled 10 s each, the ST re-sent both, the RP reported "repeat of chunk N, answering 1024 again" for each, and the copy is byte-identical to the source. A first attempt at 2 s stalls proved nothing -- no retry happened at all, because that is inside the ST's COMMAND_TIMEOUT, and the matching checksum would have been a false pass. The cartridge image is now 10,144 of 10,240 bytes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016emgLTZm7w3qZsRbukWwE4
diegoparrilla
added a commit
that referenced
this pull request
Sep 17, 2026
SD card and CLI rough edges (EPIC-15)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A reinserted SD card works again without a reset, the device says when there
isn't one instead of failing six different ways, a retried write chunk is no
longer written twice, and the CLI reports a dropped connection instead of a
Python traceback.
A reinserted card stayed dead — two causes, not one
The driver never learns the card left.
sd_card_spi_init()begins with "Check if we're notalready initialized before proceeding" and returns early unless
STA_NOINITis set — and withcard-detect disabled on this board nothing ever sets it when a card is pulled. So on reinsertion the
driver still believed the old card was initialised, skipped its entire init sequence, and handed
FatFs a freshly powered card it had never spoken to, which failed on the first sector read.
sd_card_p->deinit()sets the bit back.FatFs never asks. While the volume stays registered,
mount_volume()never callsdisk_initialize()at all, sof_mount(NULL, "0:", 0)has to come first.Both are needed. The first version of the fix had only the unregister, and the hardware caught it:
every retry still failed with
FR_DISK_ERR.And the removal cannot be detected from FatFs results. With the card physically out,
volumeanswered
200with the cached size and free space, and a listing answered200with an emptydirectory — FatFs serves what it cached and reports no error, so the "remount on the next failure"
trigger in the plan could never fire. The main loop now asks the card directly every 2 s by reading
sector 0 through
disk_read(), which a cache cannot answer.No card: one answer, and the menu says so
volume503 no_sd_card503 no_sd_card500 disk_error503 no_sd_card500503 no_sd_cardThe menu header carries the state (
GEMDRIVE SD: mounted/SD: NO CARD) and[G]and[U]arerefused with "Insert a working microSD card: GEMDRIVE needs one." rather than launching a mode
with no drive behind it. STORY-01's retry loop clears the block about two seconds after a card is
inserted, with no reset.
The state went on the header line rather than a row of its own because a new row breaks the
menu: the dividers are at fixed pixel rows and the icons are right-aligned per header row, so one
extra line struck
ScreenmemandHookthrough, collidedIP addresswith the USB CDC heading andran the status line into
Select an option:. A framebuffer snapshot caught that.A retried write chunk was written twice
After a timeout the ST re-sends the same 1 KB chunk with a fresh token, and the RP appended it
again: the file gained a duplicate and lost its tail. Nothing on the wire distinguished a retry from
new data.
An absolute offset does not work without more state than it looks: the m68k knows only how far it
has got within the current
Fwritecall, not the file position, which the RP owns andFseekmoves — so a relative offset cannot tell a retry of a call's first chunk from the first chunk of the
next call. Instead the m68k carries a sequence number that never repeats:
write_seq, bumped onceper chunk and never per retry, sent in
d5, which the protocol always transmits and the RP wasskipping. Each open file slot remembers the last sequence it accepted and the bytes it wrote; a
repeat is answered with that same count and not written again.
The CLI no longer prints tracebacks
urllibwraps some connection failures inURLErrorand lets the rest through as bareOSErrors.Two exposures:
urlopen()itself, and the reads after a successful open — the download loop andthe
debug tailloop both calledresp.read()with no handler at all, which is where a rebootmid-transfer lands.
CONNECTION_DROPPEDnames the family once (includingIncompleteRead), andeverything maps to one
error:line and exit 8.Verified on hardware
Diego at the Atari, pulling the card and running the copies:
sector read failed -- card gone), remounted ~4 s afterreinsertion,
volumeand listings correct, 256 KB upload round-tripping identical, no reset.SD: NO CARD,[G]refused, all endpoints503 no_sd_card; blocklifts by itself on reinsertion.
logged
repeat of chunk N, answering 1024 againfor each, and the copy is byte-identical tothe source. A first attempt at 2 s stalls produced no retries at all (inside the ST's
COMMAND_TIMEOUT) and would have been a false pass.connection dropped by the device (timed out), exit 8; panic during a 4 MB upload →[Errno 32] Broken pipe, exit 8.smoke.py8/8 debug; CLI suite 119 tests (4 added), and the new tests were checked against theunfixed CLI, where all four fail with the traceback this was filed for.
Not verified
menu; the interesting case holds open handles across the pull.
same injection works on an idle device. This could be the main loop not polling under load or the
SWD injection failing to hold the pad; they need different fixes, so it is recorded rather than
guessed at. A physical press during an upload would settle it.
Also found, not fixed here
Changing
[D]rivein the menu does not change the drive the ST uses: the letter reaches the m68k inthe reply to its
HELLO, which only happens when the ST boots the cartridge. The menu showedK:while the console kept reporting
drive=C (#2)until the ST was reset. Worth its own story.The cartridge image is now 10,144 of 10,240 bytes.