feat(memory): adopt mimalloc allocator

[randomvariable]

Purpose

Adopt mimalloc for Rust-owned heap allocations while preserving native allocator relief and process-level telemetry.

Stacked on #283. Review only commits 692c81e2 and 36f6c668 until #283 merges.

Changes

• install mimalloc as the Rust global allocator in production and allocation-counted tests
• derive Rust allocator requested, committed, and retained-byte telemetry from mimalloc statistics
• run mi_collect(true) after idle eviction and from the detached allocator-maintenance thread
• retain malloc_trim(0) on glibc and malloc_zone_pressure_relief on macOS for native libraries that bypass Rust GlobalAlloc
• preserve the fleet health memory field names and byte units
• document the allocator coverage boundary
• add a realistic multi-root daemon comparison protocol for cold build, steady serving, idle eviction, and post-relief residency

Coverage boundary

mimalloc statistics cover Rust-owned heap allocations. SQLite, tree-sitter, ONNX Runtime, and other native libraries can still allocate through the platform allocator. RSS, macOS physical footprint, SQLite bytes, and subsystem estimates remain independent process checks.

Verification

• allocator relief contracts passed
• fleet health payload compatibility contracts passed
• cargo check -p agent-file-tools --all-targets
• cargo fmt --all -- --check
• git diff --check

The realistic multi-root benchmark protocol is committed without results. Both allocator arms still require paired representative-host runs before an allocator performance claim.

Scope

This PR changes allocator ownership and reclamation only. Explicit subsystem budgets, charge-before-build admission, and cache eviction policy remain separate work.

Contribution

AI-assisted implementation and verification with OpenAI Codex through Oh My Pi. The author reviewed the changes and test evidence.

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	cargo/​mimalloc@​0.1.52

View full report

[aft-alfonso]

Thank you - the stack structure is understood (reviewing only 692c81e; the deep pass holds until #283 lands, same as its own review).

One architectural question worth answering early because it shapes the whole design: mimalloc installed as the Rust global allocator captures Rust-side allocations only. A significant share of this daemon's resident footprint allocates through C malloc - SQLite page caches, tree-sitter trees, ONNX Runtime tensors - and those do not route through a Rust GlobalAlloc, so mi_collect cannot purge them. The PR removes malloc_trim (glibc) and the macOS zone relief, which are precisely the mechanisms that covered that C-side share (see #201 for the field history that motivated them). Two asks for the next revision: (1) state the intended coverage boundary explicitly - if the C-side stays on the platform allocator, the platform relief paths likely need to stay alongside mimalloc rather than be replaced by it; (2) the warm-then-idle experiment is a good start - for an allocator swap we will want the comparison run under a realistic multi-root daemon workload (cold builds + idle eviction + steady serving) with before/after RSS and phys_footprint on macOS, since MADV_FREE semantics differ between allocators and Activity-Monitor-visible numbers are part of what users report.

Also confirming: the memory telemetry fields consumed by the fleet health surface (subc/health.rs memory rollups) need to stay shape-compatible - renames or unit changes there ripple into external monitoring.