Skip to content

Name the native Flash Next checkpoint in its recipe - #705

Merged
ashhart merged 1 commit into
ashhart:mainfrom
cwschroeder:flashnext-recipe-native
Oct 13, 2026
Merged

ashhart merged 1 commit into
ashhart:mainfrom
cwschroeder:flashnext-recipe-native

Conversation

@cwschroeder

Copy link
Copy Markdown
Contributor

What this changes

The recipe's opening commands named TensorFold/Qwen3.8-Flash-Next-MLX-4bit-MTP, which the native engine refuses at load on Metal: it admits Flash Next as MLX affine 6-bit/group-32 only, as the README's model table says. The commands now use TensorFold/Qwen3.8-Flash-Next-MLX-6bit-MTP with --temperature 0, so requests that omit a temperature are served greedily as the native Flash Next server requires. The opening says the 4-bit checkpoint is refused, links the Metal runbook and #472, and marks the remaining sections as the Python 0.6.6 engine's; the CUDA setup link points at the pinned 0.6.6 instructions in cuda.md.

Receipt

Documentation only.

  • Reproduction: tensorfold-native 1.0.7 serve on a directory holding the 4-bit checkpoint's config.json and model.safetensors.index.json exits with tensorfold: Flash Next runtime kernels require the supported 6-bit affine, group_size 32 format; use a matching checkpoint.
  • The file's relative links and anchors resolve on main; git diff --check passes.

Checklist

  • No personal data, machine names, internal hosts or local paths. No AI attribution lines.
  • Documentation only: no code, arithmetic, tests or speed change.

@ashhart

ashhart commented Oct 13, 2026

Copy link
Copy Markdown
Owner

Thank you @cwschroeder, this landed in 1.0.9 as your own commit (ee9da8905), and your head is merged into the history unchanged (0f6436e42), so GitHub shows this PR as merged.

We checked it against 1.0.8. The native engine still refuses the 4-bit/group-32 checkpoint at load with the message you quote, 6-bit/group-32 is the format it admits, and both new links resolve: the runbook's Flash Next section and the Python CUDA families section. --temperature 0 is right for now. If sampled decoding on Metal (#719) lands, it stays right for greedy runs.

We'll answer your 4-bit/group-32 proposal (#706) separately.

@ashhart ashhart mentioned this pull request Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants