Repository navigation
GLM: fix sparse attention thread limit on M1 - #704
Conversation
|
I was able to reproduce this on an M2 Max: the sparse attention pipeline's |
|
Thank you, @mrv777. A pipeline that quietly allows fewer threads than its caller launches is a nasty failure (zeros, no error), and declaring the 1,024 in the generator and both kernels fixes it where it starts. Your commit landed in 1.0.9 under your name ( What we ran, against the build without your change: your sparse attention checks pass on an M3 Ultra and an M5 Ultra (3 of 3 on each). With GLM-5.3-Flash at 8 of its 45 layers, replies drafted and with |
On our M1 Max, GLM's sparse row-attention pipeline supports 832 threads, but its caller launches 1,024. Its outputs remain zero and the existing comparison test fails. This is the inherited GLM failure reported during #679 validation.
Declare the required 1,024-thread group in the generator and both generated kernels (64 and 32 heads). Extend the regressions to check pipeline capacity and compare both outputs against an independent FP64 reference. Kernel bodies, dispatch sizes and tolerances stay unchanged.
Validated on M1 Max (32-core GPU, 32 GB), macOS 26.7, Xcode 26.6, Metal 32023.883 / SDK 26.5, Zig 0.17.0:
--checkstill exits 1 for an inherited whitespace mismatch in the kernel index.Both before and after used the same local launcher selecting the installed Metal compiler/linker. Raw logs remain local. Synthetic tests only; model, performance, multi-Mac TP2 and other-hardware qualification remain unrun.