Repository navigation
Conversation
…alls them. graph_cache.Cache captures one executable graph per caller-named key and replays it when the key comes back; ranks agree on every capture, eviction and fallback through an Agree hook, and a memory floor turns misses eager. arena.Arena sub-allocates DeviceBuffer.alloc from a few large chunks (VMM or plain cuMemAlloc) through memory.setHook; borrowed spans and pointers it does not own are freed as before.
…; CUresult names fall back to a table when the driver cannot answer. cuFuncGetName is optional (CUDA 12.3+), so an older driver still opens. During the exit's teardown cuGetErrorName fails too, which printed CUDA_ERROR_DEINITIALIZED as "CUDA_ERROR_UNKNOWN_CODE (4)"; code 4 now says the process is exiting.
Use CUDA device attribute 110 for GPUDirect RDMA with VMM support. Attribute 126 reports the memory sync domain count and can incorrectly enable the RDMA allocation flag. Add a regression for the external ABI contract.
…r stays under the max, a max of 0 is eager - A capture whose agreement throws, or whose insert runs out of memory, no longer leaks its executable: the map's slot is reserved before the capture, and the executable is owned here until the insert, which cannot fail. - A host map that cannot grow counts as no room in the existing room vote, so every rank declines that miss together. - `min_keep` is clamped to `max`, so under the memory floor the cap never rises above the configured max. A max of 0 runs every step eagerly, with no collective. - A cap lowered to 0 under the floor goes back to max once every rank has room, instead of staying eager for good. - Hits make no collective: with `Agree` set, keys, settings and fingerprint changes must already match on every rank. The docs now say so: the fingerprint changes at the same step on every rank, and `fingerprintOf` is for one rank. Co-authored-by: Jonathan Regan-Beasley <197533392+jregan-beasley-smc@users.noreply.github.com>
The arena's `live` (small ranges) and `large` (own chunks) maps drop an entry on every free and gain one on every allocation for the life of the process. std's tombstones never clear, so in time they take every free slot, and each allocation, free and miss then probes the whole map. Each map now counts its removals and is rehashed in place once they reach a tenth of its capacity. Test: 20,000 random allocations and frees (small ranges and large chunks) keep a tenth of each map free at every step; without the rehash it fails.
…d RoCE collectives, plan link, fail-fast. zig/src/fabric/tp brings a rank up from TF_TP_WORLD, TF_TP_RANK, TF_TP_MASTER and TF_TP_PORT, agrees the settings digest over every rank, and gives a family all-gather, all-reduce and broadcast over NCCL (TF_COMM_BACKEND=nccl, the default), a pinned host mailbox for ranks on one host, or RoCE verbs between two hosts (TF_TP_ROCE_*), with an NCCL fallback when RoCE setup fails. Rank 0 sends each round's plan to the others over its own link (TF_TP_PLAN_*). A failure or a vanished rank ends every rank shortly after (TF_TP_FAILFAST, TF_TP_FAILFAST_GRACE_S). No family uses it yet. zig build test-tp runs the host tests; zig build tf-tp-test builds the two-GPU runner (-Dtp-nvcc= or -Dtp-fatbin= for the mailbox kernel). The RoCE all-gather adapts b12x's RoCEnante protocol.
… the stop word with no fail-fast exit; a rank killed mid-drain still fails the other fast.
… /24 the peer also has The automatic pick took the first N active RoCE ports in device-list order and never checked that the peer can reach them. On a host with two neighbours, one picked port can sit on a subnet the peer does not have, and the queue pair's RTR step then fails, which reports RoCE unavailable. Every rank now sends its usable ports' IPv4 addresses over the bootstrap. Each rank then keeps the ports whose /24 the peer also lists, ordered by subnet and each /24 no more often than both sides have it, so both ranks pick the same number. With no shared /24 the old first-N order is kept, with a warning to set TF_TP_ROCE_HCA. The exchange runs on every rank, including ranks that set TF_TP_ROCE_HCA, so the bootstrap stays in step.
…FNAME and TF_TP_MASTER are needed, with a two-host example
… kernel backends up front; run_hosts.sh passes HCA picks - `tf-tp-test kernel` reports the embedded kernel and exits 1 without one. `suite` and `failfast` on the mailbox or RoCE backend stop at once with a message that names `-Dtp-nvcc`, instead of failing later on a missing kernel. - `run_hosts.sh` stops before any step when the kernel is missing. It exports TF_TP_ROCE_HCA and NCCL_IB_HCA (derived from TF_TP_ROCE_HCA when unset), warns when a host has more than one RDMA device and neither is set, and records all three variables in host.txt.
… with another region layout are refused roce_guard.stripes computes each HCA's share of a post (its send stripe, the peer's receive range and its flag word) and refuses with error.OutOfRegion any range past its slot, its flag area or the registered region, before post_send. The verbs wire and the shm wire both post through it. The shm wire moves to roce_shm.zig and can stripe like the verbs wire (Options.shm_stripes), with an optional tap that sees every post for tests. The connection record carries the slot and registered length, so a peer with another layout is refused at connect; its magic is bumped. Tests: stripes tile the slot over two HCAs and refuse a post past the slot, a third slot, an unpadded or empty post, no HCA and a short region. On fake verbs a post past its slot reaches no verb, and a full slot stays inside the region. Over the shm wire striped as two HCAs, 2,000 variable-length gathers stay inside slot and region, bit for bit.
…would ask for one A post of about 9.6 MB faulted a DGX Spark's HCA. roce_guard.max_write (1 MiB, a constant) is checked in stripes() before the slot bounds, so the verbs wire, the shm wire and the proxy refuse a larger post with error.WriteTooLarge before any verb, whatever the slot size. Roce.init refuses a slot past it. Config.parse fails boot with a clear message when TF_TP_MAILBOX_MAX_KB would make the RoCE slot pass 1 MiB; the value is never clamped. Tests: a 9.57 MB and a 1 MiB + 16 post are refused in a 10 MiB slot and reach no verb, a 1 MiB post goes out; boot refuses TF_TP_MAILBOX_MAX_KB 1028, 2048 and 16384 and keeps 1024.
…sized ops, at most 2 MiB in flight Default off. allGatherV strides past the slot, up to TF_TP_ROCE_CHUNKED_KB, go RoCE as ceil(n / slot) launches of the tp_roce kernel: op c moves at most a slot of each shard from c x slot (RoceArgs.chunk), and an op empty on both ranks rings no doorbell and leaves the epoch. Every RDMA write keeps the 1 MiB cap. The kernel poisons the transport rather than stage past a slot, and its four failure-record blocks fold into fail_record (same stores and fences). VerbsWire.post waits for completions while the bytes posted and not completed over both HCAs would pass max_inflight (2 MiB, a constant in the post path); a wait that sees no completion for 10 s fails the proxy. Boot refuses TF_TP_ROCE_CHUNKED_KB when the kernel image lacks the tp_roce_chunk_abi marker. The setting is part of the digest both ranks agree on. Tests: chunked strides from 300 KB to a 128 MiB union posted back to back on fake verbs whose completions lag keep at most 2 MiB in flight and no write past 1 MiB, and reach the cap; the wire's own in-flight count and peak match the fake HCAs'. Over the shm wire, 2,000 chunked variable-length gathers of up to four slots arrive bit for bit.
…region; the region is DONTFORK and NOHUGEPAGE A local protection error can arrive while the IOMMU invalidates its mappings lazily. On such an error the old path left the other HCA's QP ready to send (still taking the peer's writes), hung up the fate channel without telling the peer to stop, and freed the region at once: unregistered from CUDA with no GPU sync, then QP destroy, MR deregistration and munmap. A device write after the unmap could then reach a reused page through a stale IOTLB entry. - Roce.fail and Roce.abort (the fail-fast hook) move every QP to the error state: no send goes out and no peer write lands. - Roce.deinit quiesces first: QPs to error, every posted request reaped (flushed or not), cuCtxSynchronize. A failed transport, or one that does not quiesce, keeps its region registered and mapped; the process exit tears it down. - Session.close on a failed transport goes fail-fast before closing, so the peer aborts and halts its HCAs first. - The region is MADV_DONTFORK and MADV_NOHUGEPAGE before ibv_reg_mr; a refusal fails the start. - A QP whose actual max_inline_data is under 4 B is refused (the flag post uses lkey 0 on a stack word). - One log line a rank: region base and length, each HCA's lkey and rkey, the peer's region and rkeys. Tests: on fake verbs Roce.abort halts both QPs; quiesce halts both before the first flush and reaps every posted write; an aborted transport reports itself failed, and over the shm wire the aborted rank keeps its region while the other frees it; smaps shows dc and nh on a prepared region; the inline bound refuses 3 B and takes 4.
…path on real HCAs, only on an idle link - `tf-tp-test failfast V transport [N [HOLD_S]]`: N clean 4 KiB sums, the ranks meet over NCCL, each waits for its link to go idle (every HCA's posts completed), both agree over NCCL, then the victim takes the proxy's failure path (`Roce.inject`, which is `Roce.fail`) and closes its session: both ranks halt their QPs and exit 70, and the victim's region stays registered. When either rank's link is not idle there is no inject, a clean stop, REFUSED and rc 2. - Its host-only forms (TF_TP_FAILFAST=0, or a hold) are refused unless TF_TP_ALLOW_HOST_ONLY=1 and no RDMA wire is in use (the shm wire, or no RoCE), so they never run on an HCA. - `run_transport.sh MASTER PEER_SSH OUT`, driven from rank 0's host (rank 1 over ssh): prechecks on both hosts (no GPU process, memlock unlimited), then T0 (suite), T1 and T3 (each rank as the victim). After each step it reads both hosts' kernel log since the step and every HCA *err* counter; a hit, a counter that grew, an unexpected rc or a failure line prints STOP: and exits 1. `run_transport.sh t2 OUT` runs the host-only form on one host. - The `halted` and `quiesced` lines carry each QP's state read back with ibv_query_qp (an optional symbol), each HCA's posts and completions, the completions the drain reaped and how many of them were flushes. - `Roce.allGatherV` refuses a stride past the slot (one op; the route never sends one). - The README states the RoCE limits and how to run the check. Tests: on fake verbs the link reads busy with two posts out and idle after the drain, and the report line names both QPs in the error state with every completion a flush.
The prefix index keeps one node per distinct full page of every saved entry in `by_digest`. Each saved turn adds its new pages, and each entry dropped or evicted removes the pages no other entry passes through, for the life of the server. std's tombstones never clear, so in time they take every free slot, and each save's new pages and each find's walk past the cached prefix then probe the whole map. The map now counts its removals and is rehashed in place once they reach a tenth of its capacity. Test: 3,000 turns of up to 60 fresh pages, the oldest dropped past eight, keep a tenth of the map free after every turn; without the rehash it fails.
A family can now hand the lane host a round planner (api.Rounds) that admits requests, splits prompts into pieces and runs them in the same rounds as the other streams' decode rows. The planner is opt-in: the host's own loop, with a backend's prefill_step a chunk a loop between rounds, stays the default, and a host with a planner runs the planner's loop instead (LaneHost.run picks one). The backend gains optional entries for this: begin, piece, pieces (several streams' pieces in one call), finish and join, so a round's finished prompts get their first drafts in one backend call. A planner's prompts open through lanes/open.zig like the whole and the chunked pass (open.openedWith; their first drafts sized by the head's reach under node probabilities), and a prompt cancelled at its final releases its lane as a cancelled chunk does. The lane core also takes a family row policy (lanes/trim.zig) in place of the built-in allocation, proposers that price their own drafts, and a timed switch for backends without round timers. Under a planner the host still steps the learner while no request waits or runs, and a failed piece or piece run fails only its own requests. A planner over a backend without prompt pieces (such as the external drafter wrapper) is refused at start with error.PiecesUnsupported. The round loop lives in lane_rounds.zig, a request's stream setup in lane_stream.zig and the pieces' steps in lanes/pieces.zig, keeping the host and the engine under the line limit.
…oundary, and every request queued, filling or running fails with the reason. Opened.halt (optional) lets a server stop an engine between rounds instead of under a running one; LaneHost.halt is that hook for lane-host families. A request still filling its prompt a chunk a round fails too, with its lane freed, and a prompt pass the halt cancels mid-chunk reports the halt's reason rather than a cancel.
…s waits time out The second prompt was submitted after start(), so a loaded scheduler could let the first finish its pieces before the second arrived: no queued admission, and the test's waits > 0 check failed. Both are queued now before the first round. Each wait gives up after 10 s, and a timed-out wait stops the host thread before the test's frame is gone.
family.zig is an optional vtable a family installs in place of the chat template: prompt rendering, thinking and effort fields, a streaming reply reader that parses its own call markup, and the turn it remembers. Without a family the server runs as before. chat.zig's reply collection moves to chat_generation.zig and openai.zig's run to openai_run.zig (pure moves), so neither file passes 600 lines with the hooks in.
…pt loop ends, and again if it exits with connections open.
New generation requests get 503 with Retry-After (code server_restarting; Anthropic overloaded_error) and /health answers "draining" without asking the engine. Requests in progress get TF_DRAIN_S seconds (default 20; a second signal cuts it short), then the engine's optional halt ends its rounds at a boundary and what is left fails with "server restarting: retry shortly". The engine closes before the server is freed, once.
…in check refuses the request (503) and the drain waits for it. A tests-only seam (routes.counted_hook) delivers SIGTERM in that gap; with the drain check moved before the count the test fails.
…ery TF_SSE_KEEPALIVE_S seconds (default 15).
Queueing and a long prefill send nothing after the head, and clients or proxies
that time out a silent body leave and retry from zero. Chat, completions and
responses streams get an SSE comment (": keepalive"); Anthropic streams get its
ping event. A keepalive that cannot be written cancels the request as a client
that left.
…ain checks are done The listener stays open only while a generation runs. Case 1 streamed a "@short" reply (about 180 ms) and then sent a messages request and a /health request expecting 503; when the reply ended first, the drain finished and those requests found the listener closed. The stub engine's "@Hold" reply now waits for the test's release.
…TP weight plan and its named tensors. zig/src/families/deepseek_v41 starts with the parts that need no GPU: the text config (config.json's text_config, with the per-layer CSA2 topology), EXL3 group geometry and a rank's TP slices, the pack's tensor names and shapes, the pack folder read from its safetensors headers alone, the rank's weight plan, its host images and the weights as named tensors in the reference loader's forms. zig build test-dsv41 runs the host tests on the release config, a synthetic pack and a 2.8 bpw pack's expert widths; TF_DSV41_HEADERS adds real header dumps.
… cached on disk for a fast boot. prepare.zig builds the run-time forms from a rank's named host tensors: linear strips and the routed and shared experts' tables; its test checks the dense lanes layout at every width against a reference (fixtures/lanes-ref.bin). prepared.zig writes and reads built layer images under TF_DSV41_PREPARED (_WRITE builds them, _VERIFY refuses a damaged payload), keyed by the loader sources that build them, so a changed loader never reads an older folder.
…ow's tokens, a rank's heads. engram_host.zig lays out the Engram tables of the config, hashes each window's token n-grams into table rows as the reference hasher does (checked on the release config and tokenizer, fixtures/engram-hash-ref.bin), splits the heads over the TP ranks and builds the synthetic index rows the tests use.
…ed phase (TF_DSV41_PHASES). TF_DSV41_PHASES=1 times each named phase of a round on the host clock, =sync also waits for the stream at each boundary, and TF_DSV41_PHASES_OUT=<file> writes the totals as JSON. Off by default; nothing reports into it yet.
…ack/09b-forward Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…to stack/10-prefill Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…o stack/11a-sampling zig/build/cuda.zig: the keyed sampler's device step beside main's runner block. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…nto stack/11b-draft The family drafter builds on the shared lane core from ashhart#861 unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… stack/11c-device-drafts dspark_dev.zig fills main's cuda.abi.Memcpy2D (src, src_pitch, dst, dst_pitch, width, height): the runtime layer now keeps main's copy of that ABI struct instead of a second one with CUDA's field names. Same 128-byte layout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…) into stack/11d-batched-rows Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… into stack/12a-serving-host ashhart#663 now keeps drain and keepalive on the template path, and the review moved the family hook here (ashhart#702). This layer adds that hook back: family.zig, chat_generation.zig and openai_run.zig, on top of current main's server: - Recall, /v1/imprint, System One, vision input (normalizeWith, api.Image, cache_key), Stale refusals, token_sha256, and reply_text.Shown all keep main's versions. A family's own prepared images are `family_images`, kept apart from main's `images`. - A family render now gets the prompt cache's rewind too: rewindLen probes through renderIds. That is the gap the ashhart#663 review named. No served-family gate has run on it yet. - zig/build/server.zig: `addTo` also takes the tokenizer and lanes, for `dsv41_serve`; cortheon_fuzz, hip and xpu pass them. test-cuda-host keeps main's goldens, tokenizer, GLM and tool-parse tests, and also compiles tensorfold-native's root. - Every server file imports the named `json` module (the json.zig-in-two-modules failure ashhart#663 hit). - DeepSeek.load writes a missing tokenizer's diagnostic into the caller's problem arena instead of gpa. A new test in deepseek_test.zig leaks with the old allocation and passes with the fix. - --recall with DeepSeek-V4.1 is refused at startup: Recall keys its record on the chat template's tokenizer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ashhart#637 made a sessions budget of 0 mean "no idle residency" and null mean unlimited. This layer passed its TF_DSV41_SESSION_RESIDENT / _RESIDENT_GIB defaults (0) straight through, so a session store opened without those variables would keep no idle chats. They now default to null; an explicit 0 still keeps none. The store's policy comes from one `policy` function, and its test fails with the old default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…) into stack/12a-serving-host Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
jayleaton
changed the title
Owner
|
Thank you, @jayleaton, we will carry the tokenizer and family-adapter work with your credit into the existing shared server path; the landing order and remaining checks are in our reply on #727. Posted by Ash's AI Assistant Synthena |
Open
newNode and place reserve the maps, the slot lists and the free lists before they change anything. A failed insertion no longer consumes a free-list ID or leaves an appended slot undefined (deinit read that slot's flag). The free lists keep room for every slot, so remove and prune never allocate or drop an ID. A failed insert or insertFrom prunes the path nodes it made. A failed setRam in insert drops the entry instead of leaving it indexed, and setRam returns the token blocks it took. prefix_fail_test.zig fails each allocation of a RAM insert and of a disk insert into an empty index in turn. Each time, the index must keep its shape and slot accounting, the same insert must then succeed, and deinit must leak nothing. The tests run in the Linux and the portable (macOS) session roots. Spark receipt: lane job doug-637-prefix-01 on dgx-01 (aarch64, Zig 0.17.0), 2026-10-17T02:08:30Z, VERDICT PASS, candidate tree 1322aee. zig fmt --check clean; lean 517/517 against merge-base 46a08c1, 0 new; zig build 16/16; test test-kernel-contract test-golden fuzz-cortheon 99/99 steps, 1499/1507 (8 skipped); test-sessions 3/3. With only the new tests on 9266f3f, both fail at the node-slot accounting check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… free allocLocked reserves room in `live` before it takes a range. A failed insertion used to pop a freed range and then lose it: the range was in neither the free list nor the live map. Both small-path inserts now cannot fail. A failure leaves the free list, the cursor and the chunks as they were, and the retry gets the same range. The new test spends the map's spare room so that the reuse's insertion must grow it, fails that allocation, and checks range, cursor and chunk conservation, the retry, and a clean deinit with every chunk released. Not in this change: tying memory.setHook / currentHook installation to the family that owns the arena. Jonathan Regan-Beasley's capture-ownership fix (5826f89) is unchanged. Spark receipt: lane job doug-635-arena-01 on dgx-02 (aarch64, Zig 0.17.0), 2026-10-17T02:10:06Z, VERDICT PASS, candidate tree 663ff7b. zig fmt --check clean; lean 517/517 against merge-base 46a08c1, 0 new; zig build 12/12; test test-kernel-contract test-golden fuzz-cortheon 94/94 steps, 1430/1437 (7 skipped); test-cuda-host 39/39. With only the new test on abcad7d, it fails (the freed range left its free list). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ck/03-tp-fabric Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…x fixes) into stack/04-sessions Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… fixes) into stack/07-server Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ixes) into stack/08a-config Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fixes) into stack/08b-engram Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fixes) into stack/08c-kernels Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… fixes) into stack/08e-block Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ixes) into stack/08r-runtime zig/src/cuda/arena.zig: both sides appended tests; both are kept (this branch's fallback and VMM tests, then the reuse-insertion test). The fix's reservation lands in this branch's allocLocked unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ixes) into stack/09a-kv Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ixes) into stack/09b-forward Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ixes) into stack/10-prefill Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ixes) into stack/11a-sampling Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ixes) into stack/11b-draft Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ixes) into stack/11c-device-drafts Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ixes) into stack/11d-batched-rows Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ixes) into stack/12a-serving-host Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
DeepSeek-V4.1 serving side: the release tokenizer, chat encoding, DSML tool calls, and the server's family hook that serves them over HTTP. Depends on #699. This cumulative PR targets
main; its net change over #699 (stack/11d-batched-rows) is 42 files.The server's family render / reader / commit hook lands here (moved from #663 per review). A checkpoint whose
config.jsonhasmodel_type: deepseek_v41serves through it; every other family stays on the chat-template path unchanged.Built on current
main(Recall,/v1/imprint, System One, vision input,Stalerefusals,token_sha256,reply_text.Shownare main's). Changes beyond the originalf5db10bf:prompt.zig: a family render now also gets the prompt-cache rewind (rewind_len), which the template path already had. A family's own prepared images travel asfamily_images, apart from main'simages: []api.Image.deepseek.zig:DeepSeek.loadwrites the missing-tokenizer.jsondiagnostic into the caller's problem arena instead of the general allocator. This fixes the allocation leak the previous description listed as unresolved.main.zig:--recallwith a DeepSeek-V4.1 checkpoint is refused at startup. Recall keys its record on the chat template's tokenizer, which this family does not load.anthropic.zig: a streamed tool value is copied before the reply text under it grows. This restores the hook commit's original line.zig/build/server.zigaddTotakes the tokenizer and lanes fordsv41_serve, andcortheon_fuzz,hipandxpupass them.test-cuda-hostkeeps main's goldens, GLM and tool-parse tests, and also compilestensorfold-native's root.3f392a68brings DeepSeek-V4.1 L11d: batched rows, slots and round execution #699's fix:TF_DSV41_SESSION_RESIDENT/_RESIDENT_GIBdefault to unlimited (null). sessions: session memory for paged-KV families (page pool, prefix trie, RAM residency budget, streamed NVMe tier) (DeepSeek-V4.1 stack 4/14) #637 made0mean "no idle residency".Commits in this layer:
f5db10bf(Jay Leaton), and the mergesa0d81817and3f392a68.Receipt
3f392a681f3bae7422ea19407a6ecc74df079d5f, Linux x86_64, Zig 0.17.0. No GPU or checkpoint.3f392a68):test-server-cpuwith the new test indeepseek_test.zig:3f392a68: zig, fmt, desktop, mobile: SUCCESS.mainhas since moved to0a591781; that change touchesdocs/community-benchmarks.mdonly, which this branch does not touch.-Dnvcc.Checklist
"draft": false, a resumed prompt equals a fresh one, and concurrent equals solo.Merge Danger
Door: two-way. Reverting restores #699 without a data migration.
Blast Radius: server. Only checkpoints with
model_type: deepseek_v41take the family path, and other families keep the template path. The shared server files change for everyone (thejsonmodule wiring, prompt and generation hooks behindsrv.family), so a regression there would reach every family's HTTP serving.