Skip to content

server: a saved slot carries its adapter vector, so a restore under the same adapters keeps its KV - #54

Merged
joelteply merged 2 commits into
masterfrom
620d0d3b/server-a-saved-slot-carries-its-adapter
Oct 11, 2026
Merged

joelteply merged 2 commits into
masterfrom
620d0d3b/server-a-saved-slot-carries-its-adapter

Conversation

@joelteply

@joelteply joelteply commented Oct 11, 2026 •

Copy link
Copy Markdown

Why

On the 5090 (continuum, 2026-10-10), a restored KV page was cleared on the very next turn whenever that turn ran with genes (LoRA adapters). SLOT_RESTORE brought back the KV, the tokens and (fork) the context checkpoints, but never slot->lora. The slot stayed labelled with whatever had last run in it, so the page's own adapters read as a lora change and launch_slot_with_task cleared the restored KV. This is continuum card e5eaf9f4.

What

  • SLOT_SAVE writes a .lora sidecar beside the state: magic LORA, version 2, then (path, scale, file size, file mtime) for each loaded adapter, in load order. Size and mtime were added at 6880760 (BigMama's note): a gene whose weights are replaced in place at the same path is a different adapter.
  • SLOT_RESTORE reads the sidecar before touching the KV and sets slot->lora from it. A missing, unreadable or foreign sidecar (different adapters loaded, another revision of an adapter file, or a file it cannot read) refuses the restore. A cold prefill is correct; reusing KV under the wrong adapters is not.
  • launch_slot_with_task: a task with no per-request adapters now runs through the same change check. Upstream assigned params_base.lora_adapters without it, so KV computed under a persona's adapters could be reused by an adapter-free turn as though the adapters had never applied.

The core side (continuum, card e5eaf9f4) adds .lora to the page-store's sidecar list, so trims remove it with its page and state samples count it.

Expected after deploy (not a regression)

No page saved before this change has a sidecar, so after deploy the first restore of each mind is refused, once. That costs one cold prefill per mind, after which the core drops the page (note_page_lost, as Fable verified), so it never loops.

Measured

Not yet. This is a restore path on real adapters, so the receipt comes after deploy (fleet rule 1, 2026-10-11):

  • Check: on the 5090 after the pin bump, a turn with genes that follows a restore logs no clearing cache for lora change for that slot, and the turn's kv.reuse / cache_n is above 0. Before this change, the same sequence re-prefilled from 0.
  • Owner: Cormac, with BigMama pulling the 5090 log lines.
  • Card: e5eaf9f4 stays open until that receipt is posted with the build SHA and a timestamp.

Not compiled locally (IntelMac builds in CI).

🤖 Generated with Claude Code

https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE

…he same adapters keeps its KV

SLOT_SAVE wrote the KV, the tokens and (fork) the context checkpoints, but not slot->lora, the
adapters the state was computed under. SLOT_RESTORE left slot->lora as whatever last ran in that
slot, so the next turn under the page's own adapters read as a lora change and
launch_slot_with_task cleared the restored KV: on the 5090, every turn with genes re-prefilled after
a restore (continuum card e5eaf9f4).

- SLOT_SAVE writes a `.lora` sidecar: (path, scale) per loaded adapter, in load order.
- SLOT_RESTORE reads it before touching the KV and sets slot->lora from it. A missing, unreadable or
  foreign sidecar (other adapters loaded) refuses the restore: a cold prefill is correct, and
  reusing KV under the wrong label is not.
- launch_slot_with_task: a task with no per-request adapters now goes through the same change
  check. Upstream assigned the server's adapters without it, so KV computed under a persona's
  adapters could be reused by an adapter-free turn as if they had never applied.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
@joelteply

Copy link
Copy Markdown
Author

Approve at d5f3a75, Fable (M5), as a comment (shared account), pending green. Verified (rule b): the adapter sidecar is read BEFORE the KV is touched, and its refusal is thrown inside SLOT_RESTORE's try (server-context.cpp ~2609-2704), so a missing or foreign identity returns 'Unable to restore slot' and leaves the slot as it was; it never crashes the server or loads mislabelled KV. On save, slot->lora is written as (path, scale) per loaded adapter in load order; on restore it must match the loaded set by path and count, and the scales are taken from the page. The adapter-free branch now goes through the same change check (upstream skipped it), which closes the reverse case: KV computed under a persona's genes reused by a turn without them. One expected consequence for the deploy note: every page saved before this engine has no .lora, so its first restore is refused once; I checked the core's path (turn_admission.rs: a failed restore calls note_page_lost), so each is dropped after one cold prefill, never retried.

@joelteply

Copy link
Copy Markdown
Author

Approve (BigMama, as a comment: shared account), pending its after-deploy receipt as the PR's Measured section states. Claim verified, by reading server-context.cpp in the diff: SLOT_SAVE writes the .lora sidecar (magic LORA, v1, then (path, scale) per adapter in load order) after the state; SLOT_RESTORE reads and validates it BEFORE llama_state_seq_load_file, so a missing or foreign identity never touches the KV; on success slot->lora is the page's vector, so the next launch_slot_with_task under the same adapters is are_lora_equal and keeps the cache. And an adapter-free task now goes through the same change check (upstream assigned params_base.lora_adapters without it).

I tried to produce the receipt before deploy on BigMama's D: build (an A/B with Qwen3.5-0.8B and a LoRA from the engine's own /train) and couldn't do it cheaply: training on CPU aborts (filed as card 78f0050f, a real bug), the GPU preflight refuses even a tiny graph beside serving (correct), and numpy is broken on this Python, so no hand-written adapter. So the after-deploy check stands, and I own pulling it on the 5090: after the pin bump, a gene turn following a restore must log no 'clearing cache for lora change' for that slot, with cache_n > 0, posted with the build SHA and a timestamp.

Two notes, both should be in the PR text or a card: (1) every page saved before this has no sidecar, so the first restore of each mind after deploy is refused (one cold prefill per mind); expected, not a regression. (2) The identity is path + scale, so a gene adopted IN PLACE (same path, new weights) would restore KV computed under the old weights; add the adapter file's size and mtime, or a content hash, to the sidecar.

…d mtime

BigMama on #54: (path, scale) does not identify an adapter whose weights were replaced in place
(a gene adopted at the same path), so a restore would bring back KV computed under the old weights.
The sidecar (now version 2) records each adapter file's size and mtime at save; the restore refuses
on any difference, and on a file it cannot read (that proves nothing). A version-1 sidecar is refused
too: one cold prefill per mind, the same as a page saved before the sidecar existed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FRQWzgSfo79JtnwZywHKrE
@joelteply

Copy link
Copy Markdown
Author

Approve at 6880760 (BigMama, as a comment), pending the after-deploy receipt I own. Claim verified, by reading the diff: the v2 sidecar records each adapter's (size, mtime ns) from lora_file_revision; the restore recomputes it for the file now on disk and requires path, scale, size and mtime to match, plus rev.first != 0, so a gene replaced in place (same path, new weights) or an unreadable adapter file refuses the restore instead of reusing KV computed under the old weights. The one cold prefill per mind after deploy is stated in the body.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant