Repository navigation
DeepSeek V4.1: the family — configuration, Engram tables, resident weights and the Metal operators - #825
DeepSeek V4.1: the family — configuration, Engram tables, resident weights and the Metal operators#825akol1 wants to merge 14 commits into
Conversation
|
The desktop check's single failure — |
|
Status on this head ( I cannot re-run workflows on this repository (no admin rights). A re-run of Separately, GitHub reports the branch as conflicting with current |
…he release's tf-dsv41-token-map derives the compressed key map from the original's tokenizer.json and inference config with no Python step: decode each id, fall back to the raw token when the decoder emits U+FFFD, normalize (NFKD, strip category-M marks, lowercase, the whitespace replaces, strip — the Rust chain NFKC, NFD, StripAccents, Lowercase, replaces, strip composes to this), dedupe first-seen. The multipliers come from numpy's PCG64 exactly: SeedSequence's hashmix pool, the XSL-RR output and Lemire's bounded draw, checked against the release's recorded words. The generated nfkd_data.zig comes from unicodedata 16.0.0 (the version unicode_data.zig pins); the generator stays in the fork. The golden test is the release's own 517,680-byte engram-token-map.bin, reproduced byte for byte.
… staging Config parsing, the Engram hash layout, table plans over the original shards' embed weight and scale, and the EXL3 resident weight staging behind paced reads. Host tests cover exact integer boundaries, table selection and the staged bytes. On a 512 GiB Mac the resident payload is about 385 GiB; freeBytes counts reclaimable speculative pages so the load admission sees real room. No kernels; the served engine comes with the lane backend.
… qualification runner The Engram hash/commit, sparse and compressed attention (CSA2), the hyperconnection mix, routed and shared experts, the cache and rotary glue as source Metal kernels, with zig/tests/deepseek_v41_reference as the independent CPU oracle: plain reductions in fp64, no native statistics, no injected coefficients. tf-dsv41-run drives admission, teacher-forced logits, chunk seams, same-input operator replays and profiling on the resident checkpoint. The retained whole-model gate stays failed at 4.0961% NRMS against the unchanged 1% limit; operator checks are evidence, not qualification.
…-Flash's MIT inference
…ps to it Rank-split packs (jayleaton's 2.8bpw) duplicate whole-load tensors in native.safetensors beside the per-layer shards; the index's weight map decides each name's owner file, so the plan no longer refuses the pack as duplicates.
The released checkpoints store the gate f16 and its bias f16; rank-split repacks store the weight bf16 and the bias f32. The dense kernel's weight dtype and the route kernel's bias dtype already decode all three, so the loader accepts the range and the encoders follow the stored tensor.
The gate kernel decodes its q/k weights by dtype, the Gate descriptor carries it, and the loader accepts the rank-split packs' bf16 projections beside the released f32 ones.
Jayleaton's 2.8bpw stores the compressor and indexer matrices as a bf16 weight beside the rotation scales, and prefers it over the trellis copy it also ships. The exl3 module gains the dense form: a plain product kernel over the rotated rows writing the sums region the rotate_out stage reads, so the stage composition and its bit boundaries stay the ones the reference defines. The loader rides the pack's dtypes for the router gate, bias, Engram gate weights and the indexer's weights projection; the reference takes the union and names the dense apply it still owes. The 2.8bpw plan passes: 406.6 GB, 44 shards.
The fp64 oracle's dense path mirrors the quantized one's boundaries: the input rotation keeps its f16 rounding, the product is an FP64 dot over the f16-rounded rotated row and the bf16 weight, and the output stages are the rotate_out contract. The reference errors until now named the missing half; the union is complete.
…ument, and the duplicate-tensor case stops logging at error level d58f281f grew Plan.add an owned-index argument and left the in-file tests calling the old shape; the duplicate-tensor path also logged at error level, which the test runner treats as a failure even where the test expects the error.
…indow kernel's brace
…ests The generated nfkd_data.zig now comes from tools/zig/nfkd_tables.py, a peer of tools/zig/unicode_tables.py: it rebuilds the table from unicodedata 16.0.0 and zig fmt, and its --check fails on drift. The table is one file with a generated header, and lean_check.py's generator-output exemption covers it, as it already covers the Flash Next generator outputs. tf-dsv41-token-map's test blocks were never reachable because the build registered the program only as an executable; a host_tests flag in the build's engine programs table makes zig build test run them, so the normalizer chain and the PCG64 draws are gated from now on. The tests' Hangul case expected a recomposed syllable; NFKD decomposes it and the release's chain never recomposes, so the expectation and a comment say so.
…-column rule Folds the remaining comment groups in the V4.1 reference, family, backend, native and Metal sources into single lines of at most 120 columns, keeping the technical content. The lean budget counts a comment group as a multi-line comment, so this keeps the tree at its pinned baseline.
645a96a to
1eccf66
Compare
|
Thanks @akol1, alongside @jayleaton's CUDA work. 1.1.1 includes the reconciled shared configuration, checkpoint descriptors, stored parameter precision and Engram table contracts. The complete Metal backend remains separate. The refreshed 1eccf66 stack's kept-dense EXL3, dtype-aware Engram/router loads, indexed-shard handling and reference/table-generator updates go through the credited 1.1.2 family carry and loader checks. Keeping this PR open: the reference repairs, real MTP and whole-model/residency gates remain outstanding, and no release binary serves DeepSeek checkpoints yet. Posted by Ash's AI Assistant Synthena |
Family stack refreshed onto the 1.1.1
|
|
Thanks for the DeepSeek V4.1 work. We reviewed the whole stack (#825, #826, #827, #1051, #1052) together against the upcoming 1.1.2 line, and it can't land as it stands. Three bugs need fixing first:
Two notes. #1051's new Engram gate passes its bound, but five BF16 words differ from the contract, so it's a bound pass rather than bit identity; its two gather layouts are bit-equal. And the stack has split into two lines: #826 then #1051 (precision), and #826's first 26 commits then #827 then #1052 (serving). #825's 14th commit and #826's last five are in neither serving PR. Since these branches were cut, 1.1.2 has moved config, load and state, the KV state, the runner and the pack core onto shared owners, so they no longer apply cleanly. What we'd suggest: fix the three bugs, pick one lineage (precision with #1051, or serving with #1052), and rebase onto 1.1.2 once it's out, keeping the current typed CLI and backend-neutral config/load/state rather than the older whole files. A small first PR that could land on its own: Leaving the PRs open meanwhile. - Synthema, Ash's AI Assistant. |
|
Thanks for reviewing the whole stack at once — the three bugs are all real, and I reproduced each one before touching anything. Bug 2 — verified and fixed. for (cells[128..], 0..) |*dst, i| dst.* = cells[i % 128];I checked this is bit-identical to the slow Bug 3 — verified and fixed. var signs: [block_size / 8]u8 = @splat(0);
for (0..block_size) |i| {
if (i % 3 == 1) signs[2 * (i / 16) + (i % 16) / 8] |= @as(u8, 1) << @intCast(i % 8);
}Producer and consumer now agree at 128/128 positions. The regression is contained to the The frozen baseline is regenerated, and you were right that it had to be. 8 of 16 One thing worth having in CI regardless. Bug 1 — I could not reproduce it, and I think it is a 1.1.2-only forward-compat item. In the tree and on On #1051's five BF16 words. Agreed, and it is a bound pass rather than bit identity. The receipt already separates the two — On the lineage. Your read matches what the history shows: On I have not run the full suite or anything on device for these two fixes — only - akol1 |
2 participants
Piece (b)+(c)'s kernel side of the #690 split, rebased onto the 1.1.0 release (
76aafcf27) and stacked on #824: the DS4.1 Flash family — configuration, Engram table staging, resident weight staging — and the Metal operators with their fp64-dot reference and qualification runner. The fp64 test harnesses follow in the next piece.What lands:
GLM_LOAD_LIMIT_GBas the opt-in.Host tests:
test-deepseek-v41-weights(13/13) and the shard-plan tests (the duplicate-tensor case, the noncontiguous-table case, the basename escapes) — green.zig build test -Dcpu=apple_m1: exit 0, 304/304 steps;zig build native -Dcpu=apple_m1: 54/54 steps. Both packs load end to end; the lanes receipt shows 0 bit disagreements between two slots and their solo runs on either pack; the oracle on this series reproduces the retained 4.0960584769% whole-model number digit-for-digit.Rebase notes: this branch resolves onto 1.1.0's rewritten shared infrastructure — the new
paced_readadmission gauge, the tokenizer/engine module ownership, and the mergedmetal.zigfamilies table — rather than carrying the older forms. One rebase-residue commit folds a duplicate counters binding and a brace. The comment-only tidy commit keeps the whole family inside the lean checker's 120-column, one-line-comment rule:tools/zig/lean_new.pyreports 502 problems against 503 on the 1.1.0 base, 0 new.Receipts and measurement logs live on the fork beside the branch; the tree carries the tests that regenerate them. #826 stacks on this branch.