Repository navigation
Back-ports from md-microfirmware-template and md-oric - #4
Merged
Merged
Conversation
A version now starts on a release/vX.Y.Z branch cut from main; each piece of work gets its own epic/NN-<slug> branch whose pull request targets the release branch after hardware verification, and main receives the release branch once, when the version is tagged. Nothing committed or pushed may cite local planning notes, including commit messages and PR descriptions; a git grep check to run before tagging is documented. CLAUDE.md also catches up with the code: the dev UUID in the build example, the CMake presets that pin MinSizeRel, the inline FBDRV_INLINE blit and userfw's 14 KB budget, fb_core1_loop as the Core 1 dispatcher, the palette slot and API, and the hello_text example as the starting point for a new app.
The 8x8 strike left row 0 blank and put the baseline on row 7, so descenders were squashed onto it: 'g' was byte-identical to 'q' and ',' to '.', and 'j', 'p', 'y' had one-pixel stubs. Every glyph moves up one row (lossless: row 0 was blank on all 96), putting the baseline on row 6 and leaving row 7 for g q p y j , and _. Those glyphs are re-authored, '_' gets ink for the first time, and the corrupt '"', '8' and '~' from the extracted asset are redrawn. Same glyph data as md-oric's font8x8.h. fb_font.c already walks all eight rows, so the renderer is unchanged. CLAUDE.md described a font6x8.h that no longer exists; it now describes font8x8.h.
Comments in the m68k sources, the Atari build scripts and the texture tool pointed at internal planning items a reader of this repository cannot look up. They now say what the code does. Where the old wording had also gone stale, it is corrected: there is no separate fbdrv.s any more (the copy is the FBDRV_INLINE macro in userfw.s), and userfw has no STE blitter path. Comments only: BOOT.BIN is byte-identical apart from the build time in the cartridge header.
The image reaches the cartridge window through the XIP stream, which moves 32-bit words, and the copy asked for length / 2 of them: with an odd number of 16-bit words the last one was never copied. The image was 4,173 words, so the ST saw 0x0000 where userfw_dummy_irq's rte should be. As in md-microfirmware-template: the copy macro clears the window itself, copies an odd trailing word on its own, falls back to a word copy for an unaligned source, and COPY_FIRMWARE_TO_RAM_MEMCPY takes words like the other copy macros. firmware.py emits the array 4-byte aligned, since the stream's address register has no bits 1:0. Debug builds check the window against the image after the copy and count the non-zero bytes past it. main.s and userfw.s end with a NOP tail, so the last words of each are harmless to drop, to relocate in whole longwords or to read past. build.sh and target/atarist/build.sh stop at the first failed step. The m68k build deletes the old header before firmware.py runs and exits 6 when none is produced, so a failed m68k build can no longer leave a stale target_firmware.h for the RP build to embed. build.sh falls back to md5 where md5sum is missing. The regenerated image differs from the previous one only by the two NOP tails, main.s's size in the cartridge header and its relocation count, and the build date.
__StackLimit, the heap's limit for the SDK's sbrk, was the end of RAM plus the 64 KB ROM_IN_RAM region, so the heap could grow into the cartridge window the ST executes from. On the device it did: the boot left the heap's break at 0x20031000, 4 KB inside the window. It worked only because those chunks were freed before emul_start wipes the window and copies the image in; any later growth would have written into the image. The limit is now the end of RAM, as in md-microfirmware-template. The cap alone leaves 9 KB of heap and the boot needs 13.2 KB (newlib grows the heap in 4 KB steps): the first settings allocation failed and the app jumped to Booster. The room comes from the ROM3 capture ring. At 32 KB, aligned to its size, it and the padding in front of it took 46 KB for traffic of one blit ack per frame and the IKBD bytes. It is now 4 KB, about 40 s of acks or 2.6 s of a mouse moved flat out, and the heap is 49 KB. PICO_HEAP_SIZE makes 32 KB of heap a link-time guarantee, so a build that leaves less fails to link instead of booting into Booster. PICO_MALLOC_PANIC=0 is defined before pico_sdk_init(), where it reaches the SDK's malloc.c: a failed allocation returns NULL instead of panicking, which FatFs needs to halve its cluster-sized buffer request. Every allocation already checks its result. The cartridge window is declared as an array of unknown size instead of a 4-byte int, so accesses past its first word are no longer undefined behaviour; the debug image is byte-identical with that change alone. On the device: the boot completes, the heap peaks at 13.2 KB with 36 KB never used below the window, and the window matches the image.
userfw sent no IKBD commands, so the mouse stayed in TOS's relative mode and its packets reached the RP's demux, which drops the header and decodes the payload as keys. On the ST a nudge of the mouse sent F8 01 00: dx = +1 is $01, an ESC press, and the menu exited to GEM. userfw now resets the IKBD itself, reads and drops everything up to the reset's answer ($F0, or $F1 on a later IKBD), waits a short settle delay and sends $12 (mouse off) and $1A (joysticks off). The reset comes first because for a while after a reset the IKBD undoes a mouse-off command, and TOS resets it moments before the cartridge code runs: without it, $12 was lost on a cold boot of a Mega ST and the mouse stayed on. The wait ends on the answer, with a timeout that only matters when no keyboard answers, so it does not depend on the CPU's speed. On the way out to GEM, $14 (joystick events) and then $08 (relative mouse) give TOS back its own setup; $14 also turns the mouse off, so the order matters. On a Mega ST: moving the mouse in the menu sends nothing to the RP, ESC exits to GEM, and the GEM pointer moves.
The example ships its own copy of rp/src/CMakeLists.txt, which apply.sh installs over the template's. It still had the SDK's defaults: malloc panicking on a failed allocation instead of returning NULL, and no link guarantee for the heap. An app started from it now gets the same PICO_MALLOC_PANIC=0, defined before pico_sdk_init(), and the same 32 KB PICO_HEAP_SIZE as the template.
The ST and the RP reboot independently, and an ST reset leaves the RP running with all its state. After an exit to GEM the command sentinel still held CMD_BOOT_GEM, so the next ST reset ran userfw, which read it and left at once; the demo dispatcher, meanwhile, sat in its exiting state and drew nothing. Only a power cycle recovered. userfw now says hello at every boot, as md-microfirmware-template's ST does, with three ROM3 reads before its setup: the TOS version's two bytes, from the ROM header, then the hello, whose low byte is the machine from the _MCH cookie. The RP's new st_session module writes CMD_NOP to the sentinel on the hello, long before the ST's VBL loop first reads it (the IKBD reset waits for the keyboard's answer in between), and flags a new session. The main loop then sends the demo dispatcher back to its menu, tearing down a demo that was running. The machine and TOS version are kept for the app. The ROM3 signalling windows are listed in cart_shared.h and mirrored in userfw.s: $FB82xx IKBD bytes, $FB84xx blit done, $FB88xx the hello, $FB89xx and $FB8Axx the TOS version. CLAUDE.md says what each side may assume about the other. examples/hello_text builds st_session.c too. On a Mega ST (TOS 1.04, reported as machine $00): reset, ESC to GEM, reset brings the menu back; a reset while a demo runs brings the menu, not the demo; ESC still exits to GEM.
The IKBD sends a byte every 1.28 ms and the 6850 holds one. userfw read the ACIA from a poll inside the unrolled blit, every 40 MOVEM iterations: nothing read it between blits, and the audio interrupts stretched the poll interval past a byte's time, so a second byte could overrun the first and a key press or release was lost. md-oric measured overruns and a stuck key with the same kind of polling. As in md-oric, the keyboard ACIA's receive interrupt (MFP GPIP4, $118) now forwards every byte as it arrives and drains the ACIA before returning, since the MFP only sees a new edge when the ACIA's IRQ line goes down. The handler saves D0 and A1 because it can fire between a MOVEM load and its store, never touches A0 (the audio cursor), and needs no end-of-interrupt write with the MFP in auto-EOI mode. The MIDI ACIA shares the line, so its receive interrupt is off while userfw runs and TOS's setting is written back on exit. The poll is gone from the blit, 296 bytes of cartridge code and about 50 us per frame. When the ACIA reports an overrun the handler also reads $FB8500; the RP counts it (ikbd_overruns(), printed in debug builds), so a lost byte is visible instead of a silent stuck key. On a Mega ST, ten seconds of hammering the keyboard with rollover and held keys: 211 IKBD bytes, every press matched by its release, no overrun.
DPRINTF blocks on the UART, and at the SDK's 115,200 baud the debug build's boot trace delayed the cartridge image by about 0.3 s. On a power-on the ST checks the cartridge about 0.4 s after the RP starts, so a debug build often lost the race and TOS booted to GEM without the app; only a reset brought the app up. md-devops saw the same, and md-microfirmware-template runs its console at 921,600 baud for it. At 921,600 baud the image is ready about 45 ms after the first trace line (it was 350 ms) and the main loop starts at about 130 ms. On a Mega ST a power-on now comes up in the app. Release builds have no console and were not affected. The example's build file gets the same setting. Serial terminals and tools/dev/console.py must use 921600.
userfw blitted the cart framebuffer every VBL and acknowledged every VBL, whether or not the RP had published a new frame. fb_publish() waited for any acknowledgement since its last publish, so when an app took longer than a frame to draw, acknowledgements piled up meanwhile and the next publish went ahead at once, at any point of the ST's blit: a torn frame. The frame counter the RP bumps after each publish was documented as gating the blit, but nothing read it. As in md-oric, the ST now blits only when the counter's low word has changed since its last blit, and only then flips and acknowledges. The RP bumps the counter as the last write of a publish and starts the next publish only after that acknowledgement, and between the two the ST reads nothing, so they cannot overlap and a publish can happen at any moment. A slow app gets fewer frames, never a torn one; an app within its frame budget still runs at 50 frames a second. The low word is read with one move.w, because a longword arrives as two bus reads with its halves swapped and could mix an old and a new value. The VBL flag moves out of TOS's _dskbufp ($4C6) into userfw's own state area at the end of screen page A, next to the last-seen counter. Stale comments in userfw.s and cart_shared.h (a 256-byte buffer, 12.5 kHz and 6.27 kHz audio) now describe the current audio path. On a Mega ST the menu and the four demos run at 50 frames a second with no visible break.
…es flash SELECT did nothing while the app ran: only held at power-on did it reach Booster. The module still carried blocking helpers and a watcher that launched and reset core 1, which here is the c2p worker, so calling them would have killed the framebuffer's second half and left core 0 hung in fb_core1_wait(). select.c and select.h are now md-microfirmware-template's: select_poll() never blocks and a GPIO edge interrupt keeps presses made while core 0 is busy. The main loop polls it; a short press restarts the RP, a press held 10 s is a factory reset (the global settings are erased, and Booster then clears every app's settings). SELECT is configured once, in main(), which keeps its jump to Booster when SELECT is held at power-on; examples/hello_text is wired the same way. reset.h is md-microfirmware- template's too: every interrupt is masked and cleared before the jump to Booster, so one arriving at the jump cannot lock the core up. The long press erases flash at run time, and core 1 must not fetch from flash meanwhile. Its parked loop called the SDK's out-of-line FIFO functions, which live in flash; it now uses the inline ones, and the disassembly of the loop shows no call out of RAM besides the job itself. Jobs are always joined before the main loop goes on, so core 1 is parked whenever select_poll() runs. On the device, SELECT pressed over SWD: 15 ms is ignored, 300 ms restarts the RP, which is back in its main loop 240 ms later. The factory reset was not pressed: it would erase the test machine's settings.
Core 0 ran on the SDK's 2 KB stack. The deepest static path from main, the boot's SD error trace, needed 2,192 bytes of this firmware's own frames plus newlib's printf, which the call-graph analysis cannot see: an overflow would have walked through the rest of SCRATCH_Y into core 1's stack, silently. As in md-microfirmware-template, core 0's stack is now all of SCRATCH_Y, 4 KB, with an MPU guard at its bottom, so an overflow is a HardFault; core 1's stack follows the same setting, all of SCRATCH_X with its own guard. The FATFS object leaves emul_start's frame: the deepest path is 1,576 bytes now, and core 1's jobs use under 100. examples/hello_text gets the same settings. commemul.c takes md-microfirmware-template's poll: a reader that falls a whole ring behind is counted in commOverruns (commemul_getOverruns()) and resynchronised instead of replaying a mix of old and new samples, and the DMA channel is re-armed before its transfer count runs out, where capture used to stop after 2^32 samples. The ring keeps this template's 4 KB. On the device the debug build boots with the guarded stacks, mounts the card, streams audio and reaches its main loop; overruns read 0 over SWD.
fatfs-sdk moves to v3.6.2 (rp/build.sh and the submodule), as in md-microfirmware-template: the old pin predates the fix that releases the card and SPI locks when a read fails, so the second disk call after the card was pulled waited forever. The .YMS audio stream reads the card every frame, so pulling it would have frozen the main loop within about 40 ms. On a Mega ST, with the card pulled while the menu streamed audio, the RP kept publishing 50 frames a second and took the next keys. A failed audio read is now logged once per streak in debug builds. hw_config.c is md-microfirmware-template's: sd_get_by_num() no longer accepts an index one past the end of the table, spi_get_by_num() bounds by the number of SPI buses, and the SPI mode and drive strength are named constants. settings_init() checked the flash size, the offset and the number of defaults with assert(), which every build type compiles out; they are run-time checks returning an error now. The settings dump prints <set> or <none> for any key whose name contains PASSWORD, since debug logs get saved and shared; the number formatting stays hand-rolled, keeping newlib's printf engine out of the binary. aconfig.c says never to remove a default, only to add at the end. The global defaults, which only show for a key missing from flash, match Booster's.
The link stripped every symbol from rp.elf, so a crash address could not be resolved and the SWD tools could not find the cartridge window, the counters or the build ID by name. The symbols never reach the .uf2: a release build with a fixed date is byte-identical with and without --strip-all. As in md-microfirmware-template, the ELF keeps them.
It was documented as a build target, but neither this repository nor the Pico SDK has a board header for it, so a build for it could not even configure. The builds are pico_w and pico. BOARD_TYPE_CUSTOM16MB, which only that board defined and no code reads, goes with it.
build_id.cmake writes build_id.h and build_id.c on every build, with release_build_id: the commit, then -dirty.<hash of the diff> for a tree with changes, +<type> for a CMake type other than Release and +debug for a debug build. The files are rewritten only when the ID changes. The symbol is kept through --gc-sections, so the SWD tools can read which build a device runs from its flash. Both CMakeLists, the template's and hello_text's.
md-microfirmware-template accepts pico and sidecartos_16mb too, but CI and the release build only pico_w, and rp/build.sh here builds through presets that exist for pico_w alone, so "./build.sh pico ..." never worked. Both CMakeLists now stop on any other board, and build.sh's usage, the README and CLAUDE.md name pico_w only.
As md-microfirmware-template's build scripts: - build.sh sets one RELEASE_DATE for both images, unless the caller sets one, and rp/build.sh keeps it. The cartridge header's GEMDOS date and time follow it (target/atarist/build.sh passes them to make). - The build type is release or debug, in any case; anything else stops the build. It sets DEBUG_MODE for both targets, so a release build assembles the m68k code with _DEBUG=0 instead of the Makefile's 1. The m68k sources do not use _DEBUG today, so the image is unchanged. - build.sh empties dist/ before anything else, computes the MD5 once and no longer exports VERSION: the stcmd of older atarist-toolkit-docker installers takes it as its Docker image tag. Beyond md-microfirmware-template: the SDK's binary info carried __DATE__, the day of the compile, so builds of one commit made on different days still differed. Both CMakeLists set PICO_PROGRAM_BUILD_DATE to RELEASE_DATE. Two m68k builds and two RP release builds with the same RELEASE_DATE are byte-identical, and the only date left in the RP image is RELEASE_DATE.
This template has no HTTPS build; the HTTPS_BUILD input came with the copy from md-microfirmware-template and was never set.
As md-microfirmware-template's workflows: - ARM GNU Toolchain 14.2.rel1, downloaded from ARM and cached, instead of Ubuntu's gcc-arm-none-eabi. A different compiler gives a different image. - STCMD_NO_TTY=1 instead of editing -it out of stcmd. The v1.3.0 installer's stcmd honours it, and takes its image tag from STCMD_IMAGE_TAG, not VERSION. - release and debug in a matrix, with fail-fast off. actions/checkout@v4 and setup-python@v5. - Builds are dated by their commit. A job summary gives the build ID, the ELF's size and the UF2's MD5. The release workflow gets the same toolchain, toolkit, actions and date, and keeps skipping the S3 upload for the dev UUID.
Up to now every build was MinSizeRel, because a Release build once broke at runtime and no reproducer was recorded. Both presets now set CMake Release (-O3), as md-microfirmware-template, md-drives-emulator and md-devops ship, and a debug build differs from a release build only by DEBUG_MODE=1, so it runs the same optimised code. RP_CMAKE_BUILD_TYPE overrides the CMake type in rp/build.sh, with a warning: MinSizeRel to compare against the old builds, Debug to step through the code in a debugger. The build ID carries it (+minsizerel, +cmakedebug). The hot compute files keep their per-file -O3 pragma, so they stay fast in a MinSizeRel build; their comments no longer say the global build is -Os. Measured: Release takes about 27 KB more flash than MinSizeRel. The heap keeps the same room in a release build and has 4 KB less in a debug build (45.6 KB, the ROM3 ring's alignment padding), against a 13.2 KB boot peak. Core 1's park loop still runs from RAM only, and the deepest static stack path from main is unchanged at about 1.6 KB plus newlib's printf.
The RP can refuse to start the app. st_session_veto_boot(reason) sets a boot status word; pre_auto reads it once per ST boot, before it starts userfw, and on a veto prints the reason and returns to GEM, as it does in high resolution. The reason is the app's own text (md-oric's is fixed in the m68k code): up to 125 characters, stored byte-swapped within each word as the cart bus reads it. The veto holds until st_session_allow_boot(), so an ST reset retries once the cause is gone. The status word and the message take 128 bytes after the audio buffer ($FA4500); APP_FREE moves to $FA4580. st_session_return_to_booster() leaves for Booster without a power cycle. Not reset_jump_to_booster(), which is only safe at the top of main(): it saves BOOT_FEATURE = BOOSTER in the global settings, writes CMD_RESET to the sentinel and restarts the RP, and main() then takes the jump. On CMD_RESET userfw masks interrupts (its handlers live in the cartridge, which stops answering once the RP restarts), copies a 28-byte stub into the unused tail of screen page A and runs it there: a couple of seconds' wait, so TOS scans the cartridge once Booster serves it, then a cold reset through $4, which reinitialises everything userfw took. The demo menu does this on B. Cartridge code 8334 -> 8400 bytes. Verified on a Mega ST (TOS 1.04), debug build: a veto set over SWD sent the ST back to GEM with no hello; after an RP restart the app booted; B wrote the settings, restarted the RP into Booster and the ST cold-booted into it; relaunching the app from Booster worked.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Back-ports into v1.1.0 from the sibling projects: md-microfirmware-template for the shared RP code and the build, md-oric for the framebuffer and cartridge side. Also bumps the version to v1.1.0.
The ST side and the cartridge protocol
userfw_acia_irq), no longer polled inside the blit. The poll left gaps longer than one IKBD byte, and the 6850 overran. An overrun is now reported to the RP ($FB8500,ikbd_overruns()).userfwruns. The IKBD is reset and its answer awaited before$12/$1A; without that, a cold boot sometimes lost the mouse-off. TOS's setup is restored on exit ($14,$08).st_session.c), so an ST reset after an exit to GEM runs the app again.st_session_veto_boot(reason). The ST prints the app's reason and returns to GEM instead of starting it.st_session_return_to_booster(), onBin the demo menu. It savesBOOT_FEATURE, the ST cold-resets from a stub in RAM, and the RP restarts into Booster.font8x8.hfrom md-oric: descenders have their own row, and three corrupt glyphs are fixed.memfunc.has md-microfirmware-template).Layout changes:
APP_FREEmoves from$FA4500to$FA4580(the boot status word and message take$FA4500–$FA457F). New ROM3 signalling windows$FB84xx/$FB8500/$FB88xx–$FB8Axxare listed incart_shared.h.The RP side
PICO_MALLOC_PANIC=0and a 32 KB guaranteed heap; the ROM3 ring is 4 KB and counts overruns.Build and CI
Release(-O3); debug differs only byDEBUG_MODE=1.RP_CMAKE_BUILD_TYPE=MinSizeRelgives the old build.release_build_id), readable over SWD.RELEASE_DATEfor both images; the GEMDOS header date and the SDK's build date follow it. Two builds of one commit are byte-identical.target_firmware.h.rp.elf.pico_wis the only board.STCMD_NO_TTY, dated by the commit. The release workflow uses the same toolchain.Verified on hardware
On a Mega ST (TOS 1.04), release build
8fe902e:Bto Booster, and relaunch from Booster.Also verified:
examples/hello_text(release) ran on the Mega ST.