Skip to content

feat(embed): add h.fs and a just-bash shell tool for the browser host - #616

Merged
andybons merged 6 commits into
mainfrom
feat/embed-shell-tool
Oct 14, 2026
Merged

andybons merged 6 commits into
mainfrom
feat/embed-shell-tool

Conversation

@andybons

@andybons andybons commented Oct 14, 2026 •

Copy link
Copy Markdown
Contributor

Problem

A page that embeds the wasm module cannot give the model a shell: the module has no exec, and the page has no way to reach the files of the module, either to seed them or to read what a tool wrote. The spike (scratchpad/bash-spike-report.md) showed that just-bash over the module file system works end to end with a real model.

Design

Spec: docs/architecture.md Public packages (sdk/embed: the h.fs bullet and the shell export bullet), Built-in tools (default set and the bash rule it states), CI gates (browser job rows). Design: docs/design/acp-wasm.md section 5.6.

  • h.fs: async readFile, writeFile (append), readdir, stat, mkdir (recursive), rm (recursive, force), rename. Calls go to the Worker as {fsop: {id, op, args}}, run on globalThis.fs (the file system Go uses, so there is one owner of the state), and return as {fsres} with bytes as a transferred ArrayBuffer. Errors carry the POSIX code. close() rejects every call that waits. The wire ops are the eight the Worker serves; the recursive forms are page-side loops over them. Both ends of the contract are in bridge.mjs and workerbridge.mjs.
  • run(input, {signal, fs}) of a page tool gets the same handle, so shellTool() needs no reference to h, which does not exist yet when the tools are passed to createHarness.
  • @majorcontext/harness-embed/shell: shellTool({name, load, timeoutMs, maxCommands, maxOutput}). The input is {command}; the output is stdout, [stderr], and [exit N] (an error result for a non-zero exit), each stream clipped (head and tail) as in the spike. just-bash is an optional peer dependency (>=3.6.0 <4, Apache-2.0, checked on npm). It loads by load() at the first call; src/index.mjs, bridge.mjs, and wasm.mjs import nothing of it, so a page that does not import the subpath loads nothing extra. The default load is import("just-bash/browser"), which a bundler or an import map resolves; a page without a bundler passes its own load.
  • The shell has no network option, so curl does not exist, and no Python or other runtime is enabled. The call stops at timeoutMs (10 s) or maxCommands (10000, also the loop limit). Mounts: /work (the working directory) and /tmp when the module has it. /persist is never mounted.
  • Tool name: bash by default, renameable with name. fix(harness): let a host tool replace an unallowed built-in tool of the same name #615 is merged, so the page tool named bash replaces the built-in (sdk/embed lists page tool names in the allowed tools); the row uses the default name and checks that the model request lists exactly one bash. The request double records tool names only, so the shell schema is proven by the results (jq, curl refused, the limit), not compared.
  • Node ruling: h.fs is browser-only. Under Node the wasm module runs on the real disk of the host, so a model-driven shell over h.fs would reach every file of the user (and /work may not exist). h.fs is undefined with the wasm transport under Node and with subprocess (the native binary already has bash); shellTool then fails its call with a clear message. A test of the shared suite checks this on both transports.

Semantic change

createHarness returns fs with the wasm transport in a browser. A page tool receives fs in its run options. New subpath export ./shell; new optional peer dependency. Dev dependencies just-bash and esbuild (the browser fixture bundles just-bash/browser for the page; the lock file grows). Nothing changes for a page that does not use either.

Verification

  • TDD: the browser row and test/shell.test.mjs were written first; the row failed with Failed to fetch dynamically imported module: /embed/shell.mjs (404), the node test with a missing src/shellfs.mjs.
  • Browser row the shell tool and h.fs share the work directory with the file tools (one module, one turn of 8 tool calls, tool named bash against the scripted model): write_file then shell cat and grep -c … | sort see the file; the shell writes a file and read_file sees it; jq over a JSON file that the page seeded through h.fs; curl fails with command not found; while true; do :; done stops at the command limit (exit 126); then the page reads the shell output through h.fs and runs every h.fs call (round trips, append, stat, recursive mkdir and rm, rename, ENOENT and ENOTEMPTY codes, /tmp).
  • node --test sdk/embed/test/*.test.mjs: 55 pass (the adapter against a fake h.fs, the refusal without h.fs, h.fs undefined on both transports).
  • go build ./... && go vet ./... && go test -race ./... && test -z "$(gofmt -l .)" pass.

Review fixes: the package exports @majorcontext/harness-embed/zlib-stub (alias node:zlib to it in a bundler; only gzip, gunzip, and zcat fail; the fixture bundles with this esbuild alias; Vite and webpack take the same alias but are not tested); h.fs.readFile of a directory fails with EISDIR; each shellTool keeps its own shell and limits; maxOutputSize is 8 times maxOutput; harnesstest.Request.ToolSpecs records the description and schema, and the row asserts the bash tool description and command schema; the README and the spec say that fs reaches /persist. just-bash's cat reports any read error of a directory as "No such file or directory", so the shell row asserts only exit 1 for cat /work. Copilot threads are answered and resolved.

Output limits: maxOutput (default 20000) clips what the model sees, head and tail kept. maxOutputBytes (default 8 MiB) is a separate limit on the bytes that one call writes, including to a file. The loop limit keeps the just-bash default. A row writes about 1.1 MB through tee and the model sees a clipped result while wc -c of the file shows the full size. Without the stub, rg -z on a .gz file also fails; the docs say so.

CI job times (head 432adba vs main run 38510452277):

Job main this PR
Static 45s 38s
Unit 77s 58s
Contract 62s 67s
Runtime contract 54s 53s
Gates 30s 29s
WebAssembly 27s 19s
Node wasm 83s 62s
Browser 89s 105s

Contract grew by 5 s and Browser by 16 s. In the Browser job the main run had a warm Go build cache (browser rows step 43 s); across five earlier runs of this PR and main the step has been 64 to 70 s on both, and a local run of the new row's commands in just-bash takes under 0.5 s. npm ci adds about 2 s for the dev dependencies. The Contract difference is runner variance: this PR changes no Contract-job code.

Copilot AI balanced review requested due to automatic review settings October 14, 2026 02:50
@andybons
andybons force-pushed the feat/embed-shell-tool branch from 290dcd2 to d1407ca Compare October 14, 2026 02:53

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Filesystem lifecycle and shell configuration bugs can cause hanging calls, ignored limits, and incorrect file-operation semantics.

5 open findings
What changed in this PR

Adds browser-only filesystem access and an optional just-bash shell over the WebAssembly module’s shared filesystem.

Changes:

  • Adds the h.fs page-to-Worker filesystem bridge.
  • Adds the optional shellTool() export and just-bash adapter.
  • Adds browser, Node, and end-to-end coverage plus documentation.
File Description
sdk/​embed/​test/​suite.mjs Checks Node filesystem unavailability.
sdk/​embed/​test/​shell.test.mjs Tests shell adapter behavior.
sdk/​embed/​test/​browser/​page.mjs Exercises shell and filesystem APIs.
sdk/​embed/​test/​browser.mjs Bundles just-bash for browser tests.
sdk/​embed/​src/​workerbridge.mjs Implements Worker filesystem operations.
sdk/​embed/​src/​wasm.mjs Exposes browser fs.
sdk/​embed/​src/​shellfs.mjs Adapts h.fs for just-bash.
sdk/​embed/​src/​shell.mjs Implements shellTool().
sdk/​embed/​src/​shell.d.ts Types the shell export.
sdk/​embed/​src/​index.d.ts Types HarnessFs and tool options.
sdk/​embed/​src/​bridge.mjs Implements the page filesystem bridge.
sdk/​embed/​README.md Documents filesystem and shell usage.
sdk/​embed/​package.json Exports shell and declares dependencies.
sdk/​embed/​package-lock.json Locks new dependencies.
sdk/​embed/​AGENTS.md Records shell implementation constraints.
e2e/​browser_test.go Registers the shell browser row.
e2e/​browser_shell_test.go Verifies shared filesystem behavior.
docs/​design/​acp-wasm.md Documents the shell design.
docs/​architecture.md Updates the binding public API specification.
CHANGELOG.md Records the user-visible feature.
Files not reviewed (1)
  • sdk/embed/package-lock.json: Generated file

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread sdk/embed/src/shellfs.mjs
Comment thread sdk/embed/src/bridge.mjs
Comment thread sdk/embed/src/bridge.mjs
Comment thread sdk/embed/src/shell.mjs
Comment thread sdk/embed/src/shell.mjs
@andybons
andybons force-pushed the feat/embed-shell-tool branch 3 times, most recently from 132c7da to 489b22d Compare October 14, 2026 03:14
Problem: a page that embeds the wasm module has no shell for the model and cannot reach the files of the module. Design: h.fs is an async handle on the module file system served by the Worker; the shell export builds a just-bash tool over it, with no network and with time and command limits. Semantic change: createHarness returns fs with the wasm transport in a browser. Verification: browser row, node tests.
… shell output

Problem: the default load of the shell export fails in a bundler because just-bash imports named exports from node:zlib; h.fs readFile of a directory returned empty bytes; two shell tools shared the limits of the first; the output bound applied only after a command ended; the request double could not show the tool schema. Design: the package exports zlib-stub for a bundler alias; readFile checks fstat and fails with EISDIR; each tool keeps its own shell per harness; maxOutputSize is eight times maxOutput; harnesstest records ToolSpecs. Semantic change: as described. Verification: browser row asserts EISDIR, the bash tool description and schema, and a bundle built with the alias.
Problem: a directory move turned every rename error into copy and delete; a call after close() waited forever; mkdir recursive accepted a file; clip kept the whole text for a tiny limit. Design: fall back only for ENOSYS and EXDEV; a closed bridge rejects at once; mkdir checks that an existing component is a directory; clip handles a zero tail. Semantic change: the errors above. Verification: node test for the move.
Problem: the decode test compared a request with empty tool specs. Design: ToolSpecs holds a tool only when it has a description or a schema. Semantic change: none for requests that send neither. Verification: go test -race ./harnesstest/.
…e shell

Problem: maxOutputSize was 8 times maxOutput, so a command that wrote more than 160 KB failed, even to a file, and the loop limit followed the command limit, so seq stopped at 10000. Design: maxOutput clips what the model sees; the new maxOutputBytes (default 8 MiB) stops a runaway command; the loop limit keeps the just-bash default. Semantic change: a pipeline can write a large file. Verification: browser row writes about 2 MB through tee and the result is clipped; the file has the full size.
@andybons
andybons force-pushed the feat/embed-shell-tool branch from 3d5cbaf to a8c80c7 Compare October 14, 2026 04:33
Problem: the large-file pipeline of the shell row wrote about 2 MB. Design: 30000 padded lines write about 1.1 MB, enough to pass the clip and the old byte limit. Semantic change: none. Verification: the browser rows pass.
@andybons
andybons merged commit 10afa07 into main Oct 14, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants