Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -165,7 +165,7 @@
2. When a project root becomes unbound (no active routes/channels remaining and no pending binds), the subc daemon quiesces it: marks the actor context as subc unbound, invalidates the configure generation, retires search/callgraph/semantic build receivers, cancels queued and pending artifact work, cancels all queued maintenance jobs (returning `"maintenance_cancelled"` answers, except for active `Lsp` drains which are allowed to run/finish), and discards deferred configure maintenance. Transient unbind deliberately keeps the watcher and resident artifacts warm so a host restart can rebind without a full verification scan. Receiver generation/epoch pairs prevent already-dequeued results from committing after teardown or replacement, while per-artifact publication epochs prevent superseded workers from publishing stale disk pointers. When a new route is bound, the root is reactivated, clearing the quiesced and evicted flags.
3. After the idle TTL, and only while the root still has no bound or pending route, evict root-scoped artifact handles (callgraph store, search index, semantic index, borrowed indexes, symbol data, and inspect SQLite caches) via `evict_idle_artifacts`; stop and bounded-join the watcher on a detached reaper thread; and shut down reopenable LSP clients in the background. Subsequent queries trigger asynchronous index reloads. Because edits during watcher downtime go unobserved, advance artifact publication epochs and invalidate the verify memo, forcing `WarmVerifyPlan::Strict` re-verification on a later bind. The process-wide tree-sitter parser cache and shared `aft.db` connection are not per-root resources.
4. If the unbound root directory no longer exists, remove its idle executor actor and drop its LSP, bash watchdog, channels, and registries on a detached teardown thread. Purge detached-session replay and wake state for that root; a missing-directory root cannot be rebound by the plugin. If cleanup of an idle or deleted root is blocked, a detailed reap blocker census (`ReapBlockerCensus`) tracks and exposes the specific blockers (such as active route channels, quiescing status, background bash waits, or pending/queued maintenance tasks) within the subc health report -- `crates/aft/src/subc/health.rs`.
5. Under macOS and Linux, after sweeping idle roots or periodically on transport ticks when reported allocator slack is >= 1 GiB, request memory pressure relief from the OS allocator via `relieve_allocator_pressure` to reclaim unused pages -- `crates/aft/src/memory.rs`.
5. After sweeping idle roots, request forced mimalloc collection via `relieve_allocator_pressure`. Periodically sample mimalloc statistics on the detached `aft-mem-relief` thread and collect when retained committed memory is at least 1 GiB. The SubC transport and stdin ticks only perform a cheap cadence comparison -- `crates/aft/src/memory.rs`.
6. Track process-wide and root-scoped memory usage (including SQLite allocator metrics and OS RSS memory) via memory snapshots returned in status reports -- `crates/aft/src/memory.rs`, `crates/aft/src/commands/status.rs`. Status runtime counts expose live watcher runtimes, live actor roots, and open routes. Key status memory roots by `ProjectRootId` on all platforms to prevent path-casing/verbatim comparison mismatches. To prevent large status payloads from exceeding metrics cache limits, the per-root detail breakdown in status payloads and health check metrics is capped (e.g. at the top 8 roots by attributed bytes), and the remaining entries are rolled up in a compact summarized footprint -- `crates/aft/src/subc/health.rs`, `crates/aft/src/memory.rs`.

**Codebase inspection flow:**
Expand Down Expand Up @@ -362,8 +362,8 @@
**MemoryEstimate / MemorySnapshot:**
- Purpose: Track, attribute, and report process-wide and subsystem-specific memory usage.
- Location: `crates/aft/src/memory.rs`
- Pattern: Diagnostic structures and OS memory allocator hooks.
- Contains: Subsystem memory estimation helpers, SQLite allocator query bindings (`sqlite3_memory_used`), platform-specific resident set size (RSS), and macOS kernel physical footprint (`phys_footprint_bytes` via `proc_pid_rusage RUSAGE_INFO_V4`) queries. Periodic allocator slack scans run on a detached background-priority `aft-mem-relief` thread because allocator inspection can block. Transport and stdin ticks only perform a cheap cadence check.
- Pattern: Diagnostic structures with dual-domain idle reclamation.
- Contains: Subsystem memory estimation helpers, SQLite allocator query bindings (`sqlite3_memory_used`), mimalloc committed/requested byte telemetry for Rust-owned heap allocations, platform-specific resident set size (RSS), and macOS kernel physical footprint (`phys_footprint_bytes` via `proc_pid_rusage RUSAGE_INFO_V4`) queries. Native libraries such as SQLite, tree-sitter, and ONNX Runtime can allocate through the platform allocator instead of Rust `GlobalAlloc`; mimalloc statistics therefore do not represent the full process. Idle relief runs `mi_collect(true)` plus the platform relief primitive (`malloc_trim(0)` on glibc or `malloc_zone_pressure_relief` on macOS) on the detached background-priority `aft-mem-relief` thread. Transport and stdin ticks only perform a cheap cadence check. The fleet health memory field names and byte units remain stable across allocator backends.

**FleetStatusClient:**
- Purpose: Publish AFT's project-scoped status segment to the fleet status-holder plane (`prefrontal-core`).
Expand Down
26 changes: 26 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

84 changes: 84 additions & 0 deletions benchmarks/allocator-daemon/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# AFT allocator daemon benchmark

Compare the parent system-allocator build with the mimalloc build under the same long-lived SubC daemon workload.

This benchmark is an evidence protocol. It does not contain accepted allocator results. Record results only after both arms run on the same host with the same repository roots and configuration.

## Coverage boundary

The mimalloc arm installs mimalloc through Rust `GlobalAlloc`. Rust-owned heap allocations use mimalloc. Native libraries can still allocate through the platform allocator. This includes SQLite, tree-sitter, ONNX Runtime, and other C or C++ dependencies unless their build explicitly routes `malloc` through mimalloc.

The idle relief pass therefore covers both domains:

- `mi_collect(true)` releases unused mimalloc pages.
- `malloc_trim(0)` requests glibc native-heap relief on Linux.
- `malloc_zone_pressure_relief(NULL, 0)` requests native-zone relief on macOS.

Process RSS, macOS physical footprint, SQLite bytes, and subsystem estimates remain independent checks. Mimalloc statistics do not represent the full process.

## Required arms

| Arm | Build | Purpose |
|---|---|---|
| `system` | Parent commit of the mimalloc change | Baseline platform allocator behavior |
| `mimalloc` | PR branch | Rust allocator change with dual-domain idle relief |

Build both binaries from clean worktrees. Do not compare binaries with different AFT features or root-index code.

## Required workload

Use at least seven real Git roots. Include small, medium, and large roots. Use the same absolute root paths and selected search, semantic, and callgraph indexes for both arms.

Run these phases in order:

1. **Cold build**: Clear only AFT index storage. Start the isolated SubC daemon. Wait until every selected root artifact reaches a terminal state.
2. **Steady serving**: Issue a fixed reader corpus at a fixed rate while the daemon remains bound. Include read, grep, glob, outline, and callgraph queries.
3. **Idle eviction**: Close every route. Wait for the configured idle-root eviction boundary. Confirm that the daemon reports each root eviction.
4. **Post-relief idle**: Keep the daemon alive for at least two allocator scan intervals. Do not submit new work.

Use an isolated connection file, config root, data root, and log root for each arm. Never point this benchmark at the production SubC daemon.

## Sampling

Sample at five-second intervals. Record these columns:

```text
timestamp,arm,phase,pid,rss_bytes,phys_footprint_bytes,vm_swap_bytes,cpu_percent,thread_count,open_routes,live_actor_roots,allocator_slack_bytes,allocator_slack_measured,sqlite_bytes,total_attributed_bytes
```

Linux obtains RSS and swap from `/proc/<pid>/status`. macOS obtains RSS and physical footprint from `proc_pidinfo` and `proc_pid_rusage`, matching AFT's `memory.rs` implementation. Obtain allocator, SQLite, root, and route values from the existing SubC health memory and runtime rollups. Keep field names and byte units unchanged.

Capture these events with timestamps:

- daemon ready
- each root artifact completion
- steady-serving start and stop
- each idle-root eviction
- each allocator pressure-relief log
- daemon shutdown

## Controls

- Use the same host without other build or indexing work.
- Run the arms in alternating order across at least three pairs.
- Reboot or allow the host to return to the same memory-pressure baseline before each pair.
- Keep power mode, CPU governor, semantic backend, model cache, and root revisions fixed.
- Preserve model downloads between arms. Clear generated AFT indexes between arms.
- Exclude a pair when either arm has a root failure, daemon restart, transport timeout, or changed Git revision.

## Report

Report each pair separately and then report the median difference. Include:

- peak RSS during cold build
- peak macOS physical footprint during cold build
- p50 and p99 reader latency during steady serving
- artifact build completion time
- RSS and physical footprint immediately before eviction
- RSS and physical footprint after each relief pass
- final RSS, physical footprint, and swap after post-relief idle
- allocator slack, SQLite bytes, and attributed bytes at every phase boundary

Do not use RSS alone on macOS. `MADV_FREE` can leave reclaimable pages visible in RSS after the allocator surrendered them. Physical footprint is the user-visible held-memory check for that platform.

Do not claim that mimalloc reclaims native allocations from mimalloc statistics. Attribute a reduction to the combined relief pass unless a dedicated native-allocation experiment isolates the allocator domain.
1 change: 1 addition & 0 deletions crates/aft/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ crossbeam-channel = "0.5"
parking_lot = "0.12"
portable-pty = "0.9"
libc = "0.2"
mimalloc = { version = "0.1.52", features = ["extended"] }
getrandom = "0.3"
tree-sitter = "0.26"
tree-sitter-typescript = "0.23.2"
Expand Down
4 changes: 4 additions & 0 deletions crates/aft/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,10 @@
// Response::error instead of panicking. Confirmed zero .unwrap()/.expect() in
// production error paths as of v0.6.3 audit.

#[cfg(not(test))]
#[global_allocator]
static GLOBAL_ALLOCATOR: mimalloc::MiMalloc = mimalloc::MiMalloc;

pub mod agent_child_env;
pub mod alert_records;
pub mod alert_state;
Expand Down
Loading
Loading