perf(upstream): selectively port high-ROI Hermes efficiency gains - #34
Draft
RecursiveIntell wants to merge 33 commits into
Draft
perf(upstream): selectively port high-ROI Hermes efficiency gains#34RecursiveIntell wants to merge 33 commits into
RecursiveIntell wants to merge 33 commits into
Conversation
૮ >ﻌ< ა ci reviewran on 2d90edd — chore(upstream): remove completed source-export workflow
|
(cherry picked from commit 5180601)
…y imports the SDK `cmd_dashboard` started the background MCP discovery thread before importing `hermes_cli.web_server`. The thread's first act is the ~350ms `mcp` SDK import, which holds the GIL against the main thread's own web_server import, so the HERMES_BACKEND_READY sentinel — and every renderer paint behind it — moved ~300ms later on every Desktop cold start with any MCP server configured. Desktop `serve` (headless + HERMES_DESKTOP=1) now arms discovery one second after the sentinel instead. Starting it AT the bind was measured to give back most of the gain (the renderer's WebSocket connect + first hydration reads contend on the same loop). An agent build inside that window pulls the deferred start forward itself via `wait_for_mcp_discovery`, so the bounded join and the late-binding tool refresh behave exactly as before. Dashboard and non-Desktop `serve` keep the eager pre-import ordering. Minimal reimplementation of the MCP-deferral slice of NousResearch#96751 by @helix4u; the plugin-route deferral / 503 middleware / cron-after-bind slices were measured at ~0-10ms each and are not taken. Co-authored-by: Gille <4317663+helix4u@users.noreply.github.com> (cherry picked from commit 4155ea9)
… + gate Pattern-C architectural fix (write-lock contention), completing what NousResearch#90734 started: that PR fixed the four UNLOCKED readers racing the writer connection; this one fixes the 39 LOCKED pure readers convoying every concurrent turn's persistence behind the global writer lock, and adds the gate that stops the class from re-entering. The gateway shares ONE SessionDB across every agent. A read-only query under `with self._lock:` blocks all concurrent writers for its duration; under WAL, _read_ctx() serves the same read from a pooled read-only connection with no lock at all (non-WAL falls back to the locked writer byte-for-byte, so DELETE-journal installs are unchanged). Converted (SELECT-only bodies, mechanical `with self._lock:` → `with self._read_ctx() as conn:`): gateway routing loaders, session/ message counters, titles, compression tip/lineage/cooldown readers, telegram topic bindings, prune candidate scans, meta readers, resume resolution — 39 methods, verified per-method that every statement is a SELECT and every conn use stays inside the with-block. Read-modify- write methods and checkpoint/maintenance PRAGMAs stay on the writer lock (their read is ordered against their own write). Gate: tests/state/test_no_locked_readers_gate.py — an AST scanner over SessionDB that fails CI when a pure-read method body takes the writer lock, with a sabotage self-check proving the scanner fires. On pre-conversion main it reports 38 violations; on this branch, zero. Measured (2 writer threads + 3 reader threads, 3s, WAL): before: 78.5k reads, p99 2.49ms, max 160.7ms (readers convoy) after: 200.4k reads, p99 0.53ms, max 25.5ms (2.6x throughput, 4.7x better p99, 6x better tail; writes unchanged) Honest caveat: on runtimes where WAL is refused (the currently-bundled SQLite 3.46 trips the WAL-reset-vulnerability gate → journal=DELETE), _read_ctx() falls back to the identical locked-writer path and this change is behavior-neutral by construction; the win applies to WAL installs (legacy WAL DBs, fixed runtimes, wal-configured operators). Suites: tests/test_hermes_state.py 243 passed; combined state sweep 540 passed — the only 3 reds are the pre-existing test_fts_runtime_rebuild failures, verified failing on clean origin/main before this change. (cherry picked from commit 0534f10)
…es land within 250ms Every bot-to-bot DM is a fresh `hermes -p <bot> chat -Q` process, so it pays agent startup on each hop. Profiling one hop showed the single largest controllable cost was a live GET /models against the provider on EVERY launch (0.3-0.6s normally, up to the 15s probe timeout on a slow endpoint) — the in-memory endpoint-metadata cache is per process and the Nous persistent context cache is bypassed by design so the portal stays authoritative. - model_metadata: memoize successful remote /models probes on disk (cache/endpoint_model_metadata.json) with the SAME 300s TTL as the in-memory cache, so authority semantics are unchanged (reconciliation still lands within 5 minutes) but the answer is shared across processes. Local endpoints are never memoized (LM Studio reloads). - bot_relay: the cross-machine reply waiter polls the reply file every 250ms instead of every 2s — up to 2s of dead air on every relayed reply. Nothing here changes turn ordering: DMs and group rounds stay serial. Live (polis-hermes bot, spawn -> first API request, cold, 5-6 runs): main median 1.23s (one 20.8s outlier = probe stall) -> 0.96s, no stalls. (cherry picked from commit 32fe129)
(cherry picked from commit 6064668)
(cherry picked from commit cb0b66c)
The background-process poller for non-local backends ran `cat` on the whole log file every two seconds, then threw away everything except the part it had not seen yet. The offset it needed was already tracked one line below, so the full read was pure waste. Cost of one poll grew with the total output so far, which makes the cost of a run grow with the square of its length. A job writing 10 MB over an hour moved about 9 GB across the docker or SSH channel to deliver 10 MB of output. The poller now asks the shell for the file size and the bytes after the offset in one command. Reading the size first and cutting the tail at that same size keeps the two in step, so a file that grows mid-command never sends a byte twice. A file that shrank was rotated or truncated, so the offset drops back to 0 and the buffer is dropped. The output buffer is now appended to rather than replaced, matching the local reader loops, and the offset is counted in bytes because the shell counts bytes. (cherry picked from commit 8b681f7)
`hermes_home_key` called `Path.resolve()` on every call, which is a filesystem call. `ToolRegistry.current_scope_key()` calls it, and `_merged_tools()` calls that, so every single registry lookup went to the disk: `get_entry`, `get_toolset_for_tool`, `get_emoji`, and the rest. Anything that looks up many tools in a row paid for it. Building the deferred-tool catalog for `tool_search` calls `get_entry` once per tool, so with 400 deferrable tools one search did 800 path resolutions. On Windows that is `nt._getfinalpathname`, and it took 132ms of the 148ms the catalog build spent. The tokenizing everyone would suspect first was under 4ms of it. The resolved value is now remembered per input path. A process only sees a handful of home paths, so the dict stays tiny. Measured on the same 400-tool catalog build, the lookups drop from 132ms to 3.1ms with one entry cached. A path that does not exist yet is resolved but not remembered, because the answer can change once it is created, for instance when part of the path turns out to be a link. A different home path is a different key, so switching profiles or `HERMES_HOME` gets its own entry rather than reusing a stale one. (cherry picked from commit 4a17802)
The OSV preflight is a synchronous network POST to api.osv.dev (up to 10s timeout) on every MCP stdio server start. Tools like `hermes mcp test` and MCP reconnect ladders spawn the same package repeatedly, but the previous in-process cache was empty after every process restart, so each run re-queried OSV and added 5.91x variance to the connection-time span. Persist the malware-check verdict cache to `<hermes_home>/cache/osv_check.json`. Cache expiry is stored as an absolute wall-clock timestamp, so it survives restarts and monotonic-clock skew. Loading only adds missing keys so an in-memory overwrite (e.g. a test forcing expiry) is not silently reversed by the disk copy. Writes are atomic (temp file + rename) and happen under the existing cache lock. Fixes the Hermes-owned OSV preflight variance component of NousResearch#68416. Server-side `initialize` time is outside Hermes' control. - Adds `hermes_constants.get_hermes_home()` lazy import to keep `tools/osv_check.py` import-safe (stdlib + typing only at module scope). - Switches cache timestamps from `time.monotonic()` to `time.time()` for persistence compatibility. - Updates `tests/tools/test_osv_check.py` fixture to isolate disk cache per test via `HERMES_HOME` + `tmp_path`, and adds regression tests for persistence, reload, and disk format. (cherry picked from commit 8196d40)
… server Every stdio MCP server was wrapped in its own CPython watchdog that polled getppid() every 2s to notice an ungraceful Hermes exit (kill -9, OOM, crash, force-quit), since macOS has no PR_SET_PDEATHSIG. That is a whole interpreter per server for a job that does nothing until the moment Hermes dies: 10.1 MB physical footprint each, measured on macOS/arm64. Replace the fleet of pollers with a single supervisor per Hermes process holding the read end of a pipe only Hermes writes to. Death detection becomes EOF on that pipe -- exact and instant, rather than up to a poll interval late. Hermes sends `register <pgid>` / `unregister <pgid>` as servers come and go; on EOF the supervisor killpg's whatever is still registered, which is exactly the set whose teardown never ran. A clean shutdown unregisters as it goes, so EOF then finds nothing to kill. Servers are now spawned unwrapped. The MCP SDK already starts each stdio child in its own session, so the pgid recorded for killpg is the server's own group and the existing cleanup paths reach it unchanged. That also deletes the signal-forwarding layer the wrapper needed: wrapping had put the real server in a different session from the pgid being tracked, so a graceful killpg would have hit only the wrapper. Measured on a 5-gateway host: 10 watchdogs (~98 MB) -> 5 supervisors (~49 MB). One supervisor costs about what one watchdog did (9.9 vs 10.1 MB), so the win is (servers_per_process - 1) x ~10 MB, and a process with no stdio servers now spawns nothing at all. The supervisor reads length-capped lines rather than iterating the stream: a writer that never sends a newline would otherwise grow it without bound, which it must not be vulnerable to when it is the last defense against leaked servers. Found by feeding it /dev/zero, where it reached 15 GB. Verified beyond unit coverage: a real stdio MCP server connects and its tools are discovered on the unwrapped path; with a live Hermes holding a real connection, kill -9 reaped the server, its grandchild (in the server's group, the mcp-remote `node` case), and the supervisor exited on its own. The reap tests were sabotage-checked in both directions -- a no-op reaper fails all three, while the test pinning that a cleanly unregistered server survives keeps passing -- and each wiring half fails independently when removed. (cherry picked from commit a252d4c) (cherry picked from commit 2d783a1)
`npx` resolves a package and then FORKS: it stays alive as the real MCP server's parent for the whole process lifetime while doing no work. Measured on a 4-agent host, that is ~48 MB of private memory per stdio MCP server — and it buys nothing here, because Hermes already wraps the child in its own parent-death watchdog, so npx's supervision is a second parent nobody reads. The process tree for one server is watchdog -> `npm exec <pkg>` -> node. When the package is already in npx's cache, spawn its binary directly and drop the middle process. A cache miss changes nothing: the command stays `npx`, so a cold machine still installs on first run. The swap deliberately happens AFTER the OSV malware preflight. `_infer_ecosystem` keys off the command basename being `npx`/`uvx`/`pipx`, so rewriting first makes `check_package_for_malware` return None and silently turns the gate into a no-op. Two tests pin that ordering — one behavioural, one structural — because a future edit that moved the swap earlier would disable the malware check without failing any test of either piece alone. This is also why doing it in code beats the config workaround of pointing `command` straight at the cached path: that loses the preflight. Conservative by construction — falls back to npx for a version-pinned spec (npx owns that resolution), an ambiguous or absent `bin` map, a missing or non-executable binary, an unreadable cache entry, or no cache at all. Measured on a Mac mini running four agent gateways plus a dashboard, ten stdio MCP servers total: resident `npm exec` parents 5 -> 0, tracked footprint 1531 MB -> 1324 MB across 30 -> 25 processes, free memory 464 MB -> 697 MB on a host that was actively swapping. Every MCP server kept working; tool calls verified against a live Linear server afterwards. (cherry picked from commit 11ed840)
`_record_streamed_assistant_text` grew the turn's visible text with `+=` on an attribute. Python only grows a string in place when the target is a local variable, so this copied the whole text on every delta. The loop runs once per streamed token, so a reply of length N costs about N squared in copying. A 200 KB answer arriving in 4-character deltas moves several billion characters and burns seconds of CPU in the loop the file itself calls the hottest one in the agent. The text is now kept as a list of pieces and joined when read. Reading happens at turn end and on interrupt, not per delta, so the whole turn is linear in the length of the reply. `_fire_stream_delta` used to join on every token just to ask if the text was empty. That check now looks at the parts list. `_current_streamed_assistant_text` becomes a property over that list, so the seven readers and the call sites that clear it between turns keep working unchanged. Reading does not collapse the pieces, because a delta landing between the join and the write back would be lost. Measured with 8-character deltas: adding 20k deltas to an already long text took 3.1 times as long as the first 20k before, and 0.9 times after. (cherry picked from commit 7caee28)
…scripts in the parent heap A parent that fanned out 1,320 subagents over 13h reached 2.6 GB RSS (1.9 GB anonymous heap). Every closed child AIAgent stayed reachable and still owned a copy of its full message history. gc.get_referrers on a finished child (30-child fan-out bench, evals/fanout_resource_bench.py) showed two retainers: 1. bind_subagent_parent() stored the agent strongly in the `hermes_subagent_lifecycle_parent` ContextVar. Each child binds ITSELF for its own turn, and every asyncio Handle/Future scheduled during that turn (LSP reader loops, kernel pipe transports) snapshots the Context — 56 live Contexts held 14 finished children after the bench. The ContextVar now holds a weakref (non-weakrefable doubles fall back to a closure); get_active_subagent_parent() dereferences it. 2. AIAgent.close() cleared _session_messages but not the _db_flush_scan_prefix snapshot (a `messages[:]` shallow copy taken on every successful DB flush) nor _streamed_assistant_text_parts, so the agent — kept alive by (1) — retained every message dict. close() now drops both. The delegate_task result entry never carried `messages`; a pin test confirms the per-child result JSON is unchanged. Bench (30 children / 10 worktrees, ~100 KB final replies so retention is visible): post-fan-out live child AIAgents 14 -> 0; RSS after fan-out 636 MB -> 556 MB. With the harness' tiny default replies both runs sit at ~192-194 MB (the children's transcripts were never the dominant cost there; the leaked objects were). (cherry picked from commit c96568f)
…terminals (NousResearch#90674) Ghostty (and any terminal reporting modified letters via the kitty CSI-u or xterm modifyOtherKeys protocols) sends Shift+R as a lowercase keycode with a shift modifier. keycodeToName() lowercases the printable ASCII range, so the composer received 'r' instead of 'R' — uppercase input was silently destroyed. The shift flag was parsed correctly but discarded at the input layer. Re-apply shift to a single lowercase letter in inputForSpecialSequence so the inserted text is case-restored while keybinding consumers still see the lowercase canonical name via key.name. Covers both the CSI-u and modifyOtherKeys paths. Fixes NousResearch#90663 (cherry picked from commit c3ce416)
…s on extended-key terminals (NousResearch#105493) Since NousResearch#90674 hermes-ink restores the shifted letter's case for CSI-u and modifyOtherKeys input, so Cmd+Shift+Z reaches the composer as inp 'Z' with key.shift set. The redo branch compared inp === 'z' and missed, falling through to the printable path and inserting a literal "Z". The legacy raw-byte path (ESC Z) already delivered 'Z', so the binding was latently case-sensitive on both paths. Compare with inp.toLowerCase(), matching the copy/paste chords a few lines above. Linux Ctrl+Shift+Z is unaffected (ctrl chords keep the lowercase key name). The test drives the real TextInput through renderSync with kitty CSI-u bytes (super+z, then super+shift+z) and asserts the redo lands and no "Z" is inserted; it fails on the unfixed tree with "aZ". (cherry picked from commit 6e2b8e0)
…ocal-env gate Two /simplify-code findings on the salvaged NousResearch#95160: - The native read walked every line of the whole file with per-line Python bookkeeping even after the requested window had passed, because total_lines and the trailing byte are still needed. Once lineno is past end_line, switch to chunk.count(b'\\n') for the remainder -- same result, C speed. Page 1 of a 3M-line (123 MB) file: 448 ms -> 49 ms. Parity harness native vs shell: 20 shapes (1 MiB boundary, no trailing newline, past-EOF offsets, long lines) identical. - _native_read_enabled re-implemented the LocalEnvironment check that _lsp_local_only() already does in this class (same env-None / import policy) and memoised it on self although self.env is bound once in __init__ and never rebound. Call the existing helper, drop the cache. (cherry picked from commit 8cab422)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goal
Forward-port the highest-ROI Hermes efficiency and correctness improvements that compose with Ares without replacing Ares-owned authority, permit, governed-context, receipt, effect-settlement, or staged-activation semantics.
This is intentionally not a wholesale Hermes merge. Ares diverged from shared upstream commit
8966b0a70029cb226e35c49f91d0c2208ab1d8c4; Hermes v0.21.0 alone is 865 commits ahead of that point. A blind sync would erase ownership boundaries and make review meaningless.Result of the selective port
git cherry-pick -xcommits.ares_runtime/.git diff --checkpassed on the composed port.compileallpassed acrossagent,tools,hermes_cli,gateway,cron, andrun_agent.py.HERMES_UPSTREAM_PORT_2026-09-07.md.The one-shot port workflow removed itself after producing the branch; it is not part of the proposed merge.
Efficiency gains included
The clean subset includes:
npxdirect execution to avoid an otherwise residentnpm execparent;read_fileshell probes;Upstream benchmark numbers are source-reported evidence only. This PR does not claim Ares reproduces those performance numbers until Ares-specific benchmarks are run.
Quarantined performance work worth adapting next
Several of the most attractive Hermes optimizations conflict with current Ares and should be adapted owner-first rather than discarded:
read_filefast path and associated line-count path;These are recorded as quarantined, not rejected as ideas.
Semantic work intentionally not direct-ported
These upstream clusters need Ares-specific adaptation because they intersect stronger Ares owners:
Validation boundary
The bootstrap produced a statically valid candidate, not a merge-ready claim. Repository CI/Nix/Docker must execute on the final owner-authored PR head. The PR remains draft until those checks and any owner-sensitive follow-up review are complete.
No live Ares installation, runtime activation, state migration, or external-state mutation is performed by this PR.