Skip to content

Add model-free calibrated Metal counter probe - #680

Merged
ashhart merged 1 commit into
ashhart:mainfrom
akol1:feat/metal-counter-probe
Oct 13, 2026
Merged

ashhart merged 1 commit into
ashhart:mainfrom
akol1:feat/metal-counter-probe

Conversation

@akol1

@akol1 akol1 commented Oct 11, 2026

Copy link
Copy Markdown
Contributor

What this changes

Closes #676. Adds a standalone tf-metal-counters consumer and bounded Metal timestamp recorder. Reuses gpu_profile.Class; existing split-command Trace, engine callers, lane rounds and serving paths are unchanged.

  • Same-command-buffer sampling, two-point hardware clock calibration, completion-only CPU resolution; no per-span CPU waits or CPU-timing fallback.
  • Stage-only devices rotate serial encoders with explicit fences and resource/pipeline rebinding, including hazard-untracked direct/indirect dependencies.
  • Disabled initialization allocates nothing. One command may be in flight; nesting, capacity, premature reads/reset/destruction and invalid samples fail explicitly.
  • The CLI checks independent expected FP32 bits and instrumented/uninstrumented identity across three dependent operators. Unsupported capability is reported, not represented as zero timings.

Receipt

  • Base: e7c523eb; candidate: fbe0b167. Apple M3 Ultra, macOS 26.6.2, Zig 0.17.0, Xcode Metal toolchain. No checkpoint, MLX or PyTorch dependency.
  • zig build tf-metal-counters test-metal-counters-host test-metal-counters-gpu -Dcpu=apple_m1 --summary all: 13 passed, 0 failed, 0 skipped.
  • zig build test -Dcpu=apple_m1 --summary all: 1292 passed, 0 failed, 2 skipped. Unchanged baseline: 1290 passed, 0 failed, 2 skipped. These include CUDA host checks, not CUDA GPU qualification.
  • Actual CLI --help, --capability and default run exercised. Report: status=ok, stage sampling, 4096 independent expected outputs and identical instrumented FP32 bits; in-flight read/reset/deinit rejected. Three calibrated intervals: 10.750, 13.042 and 10.333 microseconds. These include encoder/fence overhead and are not production operator or served performance claims.
  • zig fmt --check passed for changed Zig files. python3 tools/zig/lean_check.py: 534 inherited problems, same as pinned baseline; zero added findings. Checker exits 1 and is not claimed clean.
  • Read-only specialist review: no blocking findings. Complete public tree and outgoing authorship/history scrubbed; Apache-2.0 project notices retained.
  • Not run: dispatch-boundary sampling (unavailable on this M3 Ultra); Metal M1/M2/M4/M5; CUDA GPU execution; allocation/encoder-creation/GPU-failure fault injection. Failure paths were reviewed statically. This tool is Metal-only.
  • Model exactness, served decode/prompt speed, drafted/resumed/concurrent model receipts: not applicable; no model arithmetic, runtime integration or model-support claim.

Checklist

  • Existing lane/draft/scheduler/server paths unchanged.
  • No precision traded for timing; consumer outputs checked independently to the bit.
  • Capability limits and unrun hardware are explicit.
  • Lifecycle, boundaries and error cases covered by consumer-visible tests.
  • Changed source files stay below 600 lines; comments are single-line and at most 120 columns.
  • No private identities, hostnames, local paths, credentials, model-family fixtures or attribution trailers included.

@mrv777

mrv777 commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

Validated fbe0b167 on an M1 Max (32 GB), macOS 26.7, Xcode 26.6 and Zig 0.17.0.

  • All 13 dedicated tests pass with zero skips, including Metal API and GPU validation. Eight supplemental checks pass for sample reuse, ownership, cancellation and capacity.
  • Twenty CLI probes preserve all 4,096 expected FP32 output bits; 64 recordings on one recorder pass reuse checks.
  • Full suite: 1,296 passed, one known GLM sparse-attention failure (GLM: fix sparse attention thread limit on M1 #704), two Linux-only skips. Unchanged main reproduces the same failure.

Native build, server goldens (zero unexpected differences) and formatting pass; zero new lean findings (534 inherited).

Capacity note: 2,048 spans work on this M1 Max; 2,049 returns CounterSampleBufferCreationFailed, exceeding the device’s 32,768-byte sample-buffer limit. Cleanup/reuse passes, and the default 512-span allocation succeeds. Documenting this device-dependent limit would help.

Stage sampling works; dispatch-boundary sampling is unavailable on this device. Raw logs remain local.

@ashhart
ashhart merged commit 5cc99d5 into ashhart:main Oct 13, 2026
2 checks passed
jschmied pushed a commit to jschmied/TensorFold that referenced this pull request Oct 13, 2026
jschmied pushed a commit to jschmied/TensorFold that referenced this pull request Oct 13, 2026
@ashhart

ashhart commented Oct 13, 2026

Copy link
Copy Markdown
Owner

Thank you, @akol1. A probe that times GPU work with the GPU's own counters, refuses to read them before the work is done and says plainly when a chip can't sample is a sound base for timing kernels later. Your commit landed in 1.0.9 under your name (326b09f60), and the PR's head is merged into the history with no change to the tree (5cc99d560), so GitHub shows this PR as merged. #676 closes with it.

No code changed on our side. Your CHANGELOG line is in your commit as you wrote it; our CHANGELOG commit after it (db30135a2) rewrites the line in the file's usual style and links #676.

What we checked:

  • Lifecycle, by reading. Every Metal object the probe and the recorder create is released on every path, partial inits included. The in-flight refusals can't hang, because the command buffer waits on an event the CPU signals only after it has tried them, and every error path signals it first. A chip without the counters is reported as unsupported with no CPU time in their place, and nothing in the engine imports the probe.
  • Serving untouched. tensorfold-native built with and without this PR has the same bytes in every code and data segment. Only the UUID, the debug map's object name and time stamp, and the ad-hoc signature over them differ.
  • M3 Ultra. --capability reports stage-boundary sampling and no dispatch-boundary sampling. The probe printed "status": "ok" with three dependent stages (11.2, 16.0 and 7.1 microseconds of GPU time), bit identity with the untimed run and with the independently computed values, and the in-flight refusals. All 8 tests of test-metal-counters-gpu pass.
  • M5 Ultra. --capability and the probe gave the same verdicts (stage boundaries sampled, dispatch boundaries reported unsupported), and the 8 GPU tests pass.
  • Host suite. On macOS, test-metal-counters-host (5 tests) and the 2 tests in metal/counters.zig pass inside zig build test.

One limit to know about: the dispatch-boundary path hasn't run on any GPU we have, since the M3 and the M5 both report it unsupported.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Proposal: model-free calibrated Metal GPU counter probe

3 participants