fix(llm): stop CLI backends re-reading the repo and blowing the argv limit - #19
Merged
jamesaphoenix merged 1 commit intoSep 8, 2026
Conversation
…limit Two independent problems in the `claude` and `codex` CLI backends. The prompt was passed as a single argv entry. Linux caps one argument at MAX_ARG_STRLEN (128 KiB), and a group's diff plus file excerpts routinely exceeds that, so "analyze this flow" failed the spawn outright with `Argument list too long` (E2BIG) before the model was ever reached. Prompts now go on stdin, which has no such limit. Separately, the agent addendum told every spawn to "use your built-in read/search tools to inspect files, plans, and git state" even though the prompt already carries the diffs and file excerpts. Each pass therefore paid to re-derive context it had just been handed. Measured on an identical prompt, dropping that instruction and the unreachable MCP, plugin and slash-command prefix took a pass from 57,475 to 26,191 prefix tokens (-54%), with one fewer tool call and turn. Also bounds the previously unbounded metadata batch fan-out, kills child processes when their future is dropped, logs each spawn at debug level, and adds rustfmt and clippy to the devshell.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two independent problems in the
claudeandcodexCLI backends, both hit in normal use.1.
analyze this flowcrashed withArgument list too longThe prompt was passed as a single argv entry. Linux caps one argument at
MAX_ARG_STRLEN(128 KiB — this is separate fromARG_MAX, which is 2 MB and looked fine), and a group's diff plus per-file excerpts routinely exceeds it. The spawn failed with E2BIG before the model was reached:Prompts now go on stdin, which has no such limit —
claude --printreads stdin as the prompt, andcodex execdocuments-for the same. Notepass2_user_promptnever truncatesfile.diffat all (onlynew_contentgets a 2000-token budget), so a single large changed file can reach the cap on its own.2. Every pass paid to re-read context it had already been given
AGENT_ADDENDUMtold each spawn to "use your built-in read/search tools ... to inspect files, plans, and git state" — while the prompt already carries the diffs and file excerpts. Each pass re-derived what it had just been handed. Separately, the user's MCP servers, plugins and slash commands were loaded into every spawn's prefix, none of which are reachable from a--printrun.Measured against the real binary on an identical prompt:
−54% prefix tokens per pass. The addendum now states the prompt is self-contained and forbids inspection;
--strict-mcp-config --mcp-config '{"mcpServers":{}}' --disable-slash-commandsdrop the unreachable prefix.Also included
Semaphore(3)). Note this needs >60 groups to trigger at the defaultbatch_sizeof 20, so it was likely never a rate-limit contributor — included as correctness, not as the fix.kill_on_drop(true)so a cancelled command stops burning quota instead of orphaning the child.claude_cli/codex_clitargets — the prompt byte count is what makes this class of failure obvious immediately.rustfmtandclippyadded to the nix devshell; they weren't available, which is whycargo clippy -p diffcore-corecurrently fails on 8 pre-existing errors nobody could see. Those are untouched here.Verification
a_prompt_over_the_kernel_arg_limit_reaches_the_childspawns a real child with a 200 KiB prompt and asserts arrival; it fails if the prompt returns to argv.claudebinary, not assumed.Known limitations, not addressed here
run_metadata_passhas no aggregate deadline; withINTERACTIVE_CLI_TIMEOUT_SECS = 3600a stalled pass can run long. Pre-existing.detect_status()still forksclaude auth statusonce per call. Pre-existing.