Skip to content

GLM: fix KDA and router thread limits on M1 - #708

Draft
mrv777 wants to merge 2 commits into
ashhart:mainfrom
mrv777:fix/glm-kda-router-m1
Draft

mrv777 wants to merge 2 commits into
ashhart:mainfrom
mrv777:fix/glm-kda-router-m1

Conversation

@mrv777

@mrv777 mrv777 commented Oct 11, 2026

Copy link
Copy Markdown

On M1 Max, both KDA pipelines support only 512 threads, and router variants for 4/6/7/8 rows also fall below their 1,024-thread dispatch. These launches can leave outputs unwritten without a command-buffer error.

Reserve 1,024 threads in the GLM generator and regenerate both KDA kernels and the router. Arithmetic, reduction order and dispatch sizes stay unchanged. Add regressions for every router row count, both KDA head counts, values, state/cache coverage and continuity across windows.

Dependency: This branch includes #704's sparse-attention fix and uses its generator support. Merge #704 first; this PR should then be rebased to leave only the KDA/router commit.

Validated on M1 Max with Zig 0.17.0:

  • Each regression fails when its annotation is removed; all three pass with the fixes.
  • Router matches FP64 exactly; KDA passes FP64 checks and matches a legal 512-thread control byte-for-byte.
  • Fresh full suite: 1,294 passed, zero failed, two Linux-only skips; native build and server goldens pass.
  • Formatting passes; zero new lean findings (534 inherited).

Full-model inference, two-Mac TP2 and other GPUs remain untested. Raw logs remain local.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant