Repository navigation
GEMDRIVE Runner, 1KB arbitrary ROM4 storage - #20
Open
neilrackett wants to merge 29 commits into
Open
neilrackett wants to merge 29 commits into
neilrackett wants to merge 29 commits into
Conversation
…n accessory [U] runs the Runner from the cartridge's boot hook, before the AUTO folder, the desk accessories and GEM have loaded. A program it launches finds none of them: no driver from the AUTO folder, no cookie jar on TOS 1.0x, no AES for a GEM program to talk to. Testing anything that needs a booted machine meant leaving DevOps. [S] installs GEMDRIVE and the Runner's interrupt hook, then lets TOS boot on to the desktop. runner.s tells the two apart by the dispatching sentinel value, which is still in d6 when main.s calls it, and returns after its HELLO instead of entering the poll loop. main.s grows by 38 bytes and runner.s by 12. The foreground commands -- run, load, exec, unload, cd -- are served by DEVOPS.ACC, a desk accessory built from devops_acc.s by the same Makefile. GEM gives it a turn every 100 ms while the foreground program waits for events, which is when GEMDOS is never mid-call. It answers with the Runner's own report codes, so the API and the CLI are unchanged. Around a launch it does what the desktop does for a TOS program, then puts back the resolution, screen, palette and menu bar and has the desktop redraw. Its trace lines go to the debug stream rather than the screen, and its Desk menu entry opens an alert with the Runner's cwd and the last command's result. The RP embeds the accessory it was built with and writes it to the root of the GEMDRIVE folder when the user picks [S], unless an identical copy is there, so it always matches the firmware's protocol. firmware.py gains a byte array mode for this: a TOS executable ends with a zero that the word mode trims. Until the accessory reports in, the foreground commands answer 409 accessory_not_loaded. meminfo is answered by the interrupt hook, and res is refused, since changing the resolution under GEM would leave the desktop drawn for the old one. After runner reset the ST boots back into TSR mode. GEM programs cannot be launched this way: on single-tasking TOS a GEM program started from an accessory runs inside the accessory's AES slot. The accessory has been run under Hatari with EmuTOS; TSR mode has not yet been tried on hardware or on Atari's TOS.
…ater runner load and runner unload wait for the ST's reply, 10 s and 5 s, and on a timeout record a failure and answer 504. The command stayed in the sentinel, though, and the ST ran it whenever it next looked. With the cartridge's poll loop that only happens once a hung ST recovers; with the TSR-mode accessory it is ordinary, since GEM gives the accessory no turn while a program started from the desktop is running. The load that had been reported as failed then went ahead, and the program sat in memory. The handler now clears the sentinel when it gives up, unless an Advanced Runner command has replaced it since. An ST that picked the command up just before still sends its reply, which updates the state as before.
…essory With GEMDRIVE on a letter other than C:, GEM does not load accessories from it, and the README sent those users to target/atarist/dist/ for DEVOPS.ACC -- a build output someone installing the firmware does not have. [S] writes the accessory to the root of the GEMDRIVE folder whatever the drive letter, so they can copy it from there to their own boot drive. Until they do, [S] still gives them [G] plus runner reset, runner meminfo and the runner adv commands.
Two faults in the RP's answer to a read chunk. It swapped only an even number of bytes into the window the ST reads as big-endian words, so when a chunk had an odd length its last byte reached the ST as whatever the buffer held from the chunk before. Programs are loaded in 4 KB chunks, so a program with an odd file size lost the 0 that ends its fixup table, and the relocation ran on through memory: three bombs in the loader, or a program corrupted before it started. The same byte was wrong at the end of every odd-length Fread. It also sized each chunk from the whole request instead of from what was still owed, so the last chunk of an Fread over 4 KB that was not a whole number of chunks ran up to 4 KB past the end of the caller's buffer. On a Mega STE with TOS 2.06, an STDL test program of 170,595 bytes crashed in the loader every time, started from the desktop or from the Runner; the same program padded to 170,596 bytes ran and passed. md-drives-emulator already does both of these.
The fixup table follows the symbol table, and a symbol table can have an odd size: an STDL build carries one of 50,855 bytes. The loader read the table's first offset with a long access, which at an odd address is an address error on the 68000, so such a program bombed in the loader before it started. TOS's own loader runs the same file without complaint. The first offset is now built from four bytes; the rest of the table was already read a byte at a time. 10 bytes of cartridge code.
The RP copied the current path into the window unswapped, and the ST reads that window as big-endian words, so "\STDL" reached it as "S\DT". The cartridge Runner never asks for the path; the TSR accessory does, to come back to the Runner's directory before each command, and its Dsetpath of the garbled path failed.
The RP kept GEMDRIVE's open files, directory searches and current path from one ST session to the next. The handler that clears them exists, but nothing sends CMD_RESET_GEM. A program that crashes never closes its files, so after a few crashes the 8-slot table was full, and on the next boot GEM could not open DEVOPS.ACC; only restarting the RP cleared it. GEMDRIVE's HELLO, which the cartridge sends once per boot, now clears that state first.
Both meminfo paths, the Runner's and the interrupt hook's, filled the basepage field from $4F2 and called it _run. $4F2 is _sysbase, the pointer to TOS's OS header, so on a Mega STE with TOS 2.06 the field read $00E00000, the ROM. A shared routine now follows the header: from TOS 1.02 its p_run field (offset $28) holds _run's address; TOS 1.00 has no such field and keeps _run at $602C, or $873C in the Spanish version (country code 4 in os_conf). The CLI labels the field [_run], the API reference says where it comes from, and a zero no longer suggests an old TOS. 50 bytes of cartridge code. Under Hatari with EmuTOS, a copy of the routine run from a program returned that program's own basepage. The TOS 1.00 path is untested.
…Store QUESTION.TOS asks its question on screen, waits three seconds by the 200 Hz timer, writes ANSWER.TXT containing "42" to its current directory and exits with 42, so a run can be checked from the workstation without looking at the ST: runner status shows the exit code and gemdrive get fetches the file. If the file cannot be written it exits with the GEMDOS error instead. It is built like hello-debug, with build.sh in its folder. macOS Finder litters folders with .DS_Store files; they are now ignored.
docs/api.md promises Retry-After: 1 on every 503, but no_sd_card, insufficient_memory and too_many_open_files go through write_error, which added no headers. All three are worth retrying: the card is remounted within a few seconds, and memory and FatFs lock entries free up. write_error now adds the header whenever the status is 503.
Each foreground Runner command answers 503 busy while another is in flight,
but all seven call sites passed a body length of 0, so the client got
Content-Length: 0 and no error envelope; the CLI could only print
"error (503):". They now go through write_error, which sends
{"ok":false,"code":"busy",...} and, since the previous commit, keeps the
Retry-After header.
Nothing in C:\AUTO ran. After a reset, the first request GEMDRIVE saw was
the AES looking for \*.ACC: TOS never asked it for \AUTO, so an ST whose
only drive was GEMDRIVE came up without the AUTO drivers [S] exists to
load. GEMDRIVE set _bootdev to C: but left the current drive on A:.
It now also calls Dsetdrv(C:) and Dsetpath("\") when it installs as C:, as
md-drives-emulator does since its "Fixed GEMDRIVE boot not being C drive".
Checked on a Mega STE with TOS 2.06 and no other hard disk: TOS asks
GEMDRIVE for \AUTO\*.PRG and runs what it finds there.
GEMDRIVE grows by 24 bytes and still fits its 5 KB slot.
After `runner reset`, status went on reporting DEVOPS.ACC loaded until the Runner's HELLO, several seconds into the boot. A script waiting for the accessory saw it at once, sent its next command to an ST that was still rebooting, and the command was lost. The flag is now cleared when the reset is sent, and on GEMDRIVE's HELLO, which is the first the RP hears of a physical reset. Foreground commands answer 409 accessory_not_loaded until the accessory reports in again, and the docs tell scripts to wait for it.
…load from Replace the note that TSR mode had only been tried in emulation with where it has been tested: a Mega STE with TOS 2.06, with and without an ACSI2STM. Say where things load from, as seen on that machine. With GEMDRIVE as C:, GEM starts DEVOPS.ACC from C:\ with or without another hard disk. TOS runs the AUTO folder from the boot drive: C: when GEMDRIVE is the only hard disk, but a driver that loads after the cartridge can take the boot drive, and with an ACSI2STM TOS runs D:\AUTO instead.
Other SidecarTridge apps, MD/DOOM among them, go straight to the Booster when SELECT is held as the ST is switched on. DevOps only offered [X] in its setup menu, which is no help when the menu does not come up. main.c now checks SELECT before it reads any config or starts the emulation, and jumps through reset_jump_to_booster() as those apps do. The existing SELECT actions are unchanged: a tap and a 10 s hold both act on release, so the Pico restarts with the button up.
It said GEM loads DEVOPS.ACC from the root of the boot drive. With an ACSI2STM taking the boot drive as D:, GEM still starts the accessory from C:\ when GEMDRIVE is C:, so the message pointed at the wrong drive. It now says only that the accessory has not reported in and that GEM starts it once the ST reaches the desktop.
[S] made a third boot mode out of what is an option of GEMDRIVE mode. [G]
now starts it when the new GEMDRIVE_RUNNER setting is on, which is the
default: GEMDRIVE, the Runner's interrupt hook and DEVOPS.ACC, as [S] did.
With the setting off, [G] is plain GEMDRIVE, as before.
The menu goes back to [G], [U] and [X], with a Ru[n]ner row in the GEMDRIVE
block that [N] toggles; the rows, dividers and icons below it move down one
line. An install without the setting reads it as on.
The API still calls it TSR mode ("tsr": true, unsupported_in_tsr), so
scripts are unaffected. Messages and docs point at [G] with the GEMDRIVE
Runner instead of [S].
With the GEMDRIVE Runner on, [G] writes DEVOPS.ACC to the root of the GEMDRIVE folder. Turned off, [G] left it there, and GEM went on loading an accessory with nothing to serve: RAM, an accessory slot and a Desk menu entry that said it was waiting for commands. [G] with the Runner off now deletes DEVOPS.ACC from there, whatever its contents. Deleting only an exact copy of this firmware's accessory would have left the most common case behind: after an upgrade the file is the previous firmware's. The name is the firmware's anyway, since with the Runner on [G] replaces any DEVOPS.ACC that differs from its own.
The Runner handlers each spelled out their own checks: "Runner not active" in eleven places, "busy" in seven, and the accessory check in five, so a change of wording meant editing every copy. runner_require_active, runner_require_idle and runner_require_foreground (active, accessory loaded, idle) now answer them, next to runner_require_accessory. The download and upload "Another body request is in progress" replies go through write_error too, which has added Retry-After to every 503 since it learned to. The responses are unchanged.
Busy, the cwd mirror and the accessory flag were cleared by hand in three places: on `runner reset`, on GEMDRIVE's HELLO and on the Runner's HELLO. forgetStSession() now does it for all three, so the next per-boot field is one line, not three. [G] chooses its start command and shares one tail, and the relaunch tick picks its command once. The menu's dividers and status icons are placed from the section rows, so the Ru[n]ner row's three-row shift is one number per section rather than nine. The GEMDRIVE Runner setting is read as a plain bool: settings_init loads the defaults first, so the entry is always there.
GEMDRIVE's install made C:\ current with Dsetdrv and Dsetpath("\"). Once C:
is current, the Dsetpath goes to GEMDRIVE's own handler and never reaches
TOS, and GEMDRIVE's path is already \ because the RP resets it on HELLO, so
it only cost three protocol round trips per boot. The Dsetdrv, which is what
makes TOS run C:\AUTO, stays.
The Runner tested for TSR mode twice. Its cwd baseline now runs after the
TSR return instead of being skipped before it. The basepage lookup stores
straight into the meminfo struct, with one read path for the TOS 1.00 and
1.02+ cases.
Cartridge code 10206 -> 10190 bytes: GEMDRIVE has 72 bytes free, the
Runner 50.
…command The menu-bar strip buffer said "20 rows of 160 bytes" but was 4000 bytes; it is now 3200, 800 bytes less resident RAM. The largest strip any ST resolution needs is 1760. The report macro now records the command for the status alert and loads the result itself, instead of every handler doing both, and the alert's title says GEMDRIVE Runner rather than TSR mode.
runner meminfo and runner adv meminfo printed the same snapshot with two copies of the same lines; _print_meminfo() holds them. QUESTION's build.sh checked make's status after set -e had already made that impossible, and called its C toolchain vasm/vlink.
The countdown always launched [U], so a machine used with [G] came back in the Runner after every power cycle. [G] and [U] now record which one they launched in a new per-app setting, BOOT_MODE, and the countdown launches that. The setting is written only when it changes, so launching the same mode every time costs no flash write. It stays [U] until [G] is first used, so nothing changes for anyone who never uses [G]. The countdown bar names the mode it will launch: "Booting GEMDRIVE in N s" or "Booting Runner in N s".
# Conflicts: # rp/src/emul.c
GEM gives a desk accessory no events while a dialog or alert is open, a menu is pulled down or something is being dragged on the desktop, so DEVOPS.ACC cannot pick up a command until that ends, and the Runner reports busy meanwhile. Say so, and which commands time out instead of waiting: load and unload. meminfo, reset and the adv commands come from the interrupt hook and are not held up.
The debug window carries bytes from the ST to the workstation; nothing went the other way while a program runs. rom4 is 1 KB of the cartridge window, at $FA8B00 inside APP_FREE, that the workstation writes over HTTP and any ST program reads: live parameters, scripted input for an unattended test, a driver's state. DevOps gives the bytes no meaning. Programs find it through shared variables 20-22 (magic 'R4A1', address, size), published when the RP boots, so it is there in the setup menu and in [G] and [U] before the ST looks. The area is zeroed at RP boot and survives ST resets. Each 16-bit word changes in one store, so the ST never reads half of one. - GET /api/v1/rom4 reports the address, size and magic. - GET /api/v1/rom4/data returns the 1024 bytes in ST order. - PUT /api/v1/rom4?offset=N writes the body at N. - PUT /api/v1/rom4 without an offset takes [u16 offset][u16 length][bytes] frames, written in order, so a buffer and the flag that publishes it go in one request. A PUT body is at most 1024 bytes and is gathered in the connection's response buffer, so the server needs no new RAM. The whole body is checked before anything is written; a range past the end of the area is 400 out_of_range. The CLI gains `sidecart rom4 info|read|write`, and target/atarist/test/rom4 builds ROM4CHK.TOS, which shows the first 32 bytes and sends them to the debug stream. docs/api.md documents the endpoints and how to read the area from C and m68k.
The GEMDRIVE Runner, the AUTO folder from GEMDRIVE, the GEMDRIVE loader fixes, the rom4 area, the countdown remembering the last mode, SELECT at power-on, and the Runner and API fixes, in the same user-facing style as v1.1.0. The version and date are left for the release.
Author
|
P.S. Brilliant firmware, thank you! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds some of the features and fixes I've been experimenting with while developing Flashback and others.
GEMDRIVE Runner
[G]can now boot to the GEM desktop with the Runner available, so programs find AUTO-folder drivers, the cookie jar, GEM loaded and hard-disk drivers such as ACSI2STM's already in place.[N]turns it off, giving plain GEMDRIVE as before.DEVOPS.ACCdesk accessory handlesrun,load,exec,unloadandcd. The firmware installs it in the GEMDRIVE folder automatically when the runner is on, deletes it when it's off.ROM4
/api/v1/rom4,sidecart rom4) and any ST program can read while it runs, in any mode.Other changes
C:\AUTOnow works when GEMDRIVE isC:.Dgetpathbyte order.[G]and[U]was used last.SELECTat power-on to go straight to the Booster.meminfoand timed-outload/unload, plusRetry-Afterand an error body on 503s.QUESTION.TOSandROM4CHK.TOStest programs, and README, API docs and changelog updates.Tested on a Mega STE with TOS 2.06 and in Hatari with EmuTOS. Happy to split this into separate PRs if you'd prefer.