Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,22 @@ MagicQuant investigates that space. It can combine one baseline's attention reci

I build and maintain MagicQuant on the side, for free. Developing it and experimenting with quantizations has put a frankly ridiculous amount of terabytes written (TBW) on my drives! My Hugging Face storage is also creeping toward its cap, so there will eventually be more storage to fund. If this project helps you, [supporting the work](https://sayou.biz/support) helps with those costs. Anything helps and is always appreciated. ❤️

## Nonlinear wins: more fidelity for the extra bytes

A hybrid sitting between Q5 and Q6 is useful when it offers a **better trade than the normal step up**. MagicQuant draws a straight-line size/KLD comparison between neighboring survivors, then checks whether a measured hybrid beats that line. On a graph with file size on the horizontal axis and KLD on the vertical axis, a nonlinear winner sits **below the line**: less divergence than the reference trade at that size.

The original 4B campaign makes this concrete:

| Choice | Size (GB) | Measured KLD ↓ |
| --- | ---: | ---: |
| UD-Q5_K_XL recipe | 2.73 | 0.009839 |
| **MQ-Q5_K_1 hybrid** | **2.88** | **0.006632** |
| LM-Q6_K baseline | 3.08 | 0.004640 |

At 2.88 GB, the straight-line comparison gives about **0.007611 KLD**. The hybrid measures **0.006632—about 12.9% below that line**. Compared with the smaller UD recipe, it buys about **32.6% lower KLD for 5.5% more storage**. Those percentages use the rounded table values.

That is why an intermediate hybrid can earn a download slot. MagicQuant also keeps dominance wins: lower measured KLD at the same or smaller size. Its [selection rules](https://github.com/magiccodingman/MagicQuant/blob/main/wiki/docs/Nonlinear-Winners-And-Survivors.md) explain both cases; the [original worked example](https://github.com/magiccodingman/MagicQuant/blob/main/wiki/overview.md#example) preserves the full table and tensor recipes. The UD result is a controlled reconstruction of the external recipe, not a benchmark of the original uploaded artifact.

## MagicQuant in the wild

**[Explore the MagicQuant collection on Hugging Face →](https://huggingface.co/collections/magiccodingman/magic-quant)**
Expand All @@ -48,8 +64,6 @@ The [Qwen3.6-35B-A3B release](https://huggingface.co/magiccodingman/Qwen3.6-35B-

These are measured options at different storage budgets. The full release reaches down to a 9.59 GB hybrid, with a correspondingly larger divergence. The [complete table and pinned evidence](https://github.com/magiccodingman/MagicQuant/blob/main/wiki/showcase.md#discovery-a-35b-moe-release) make that trade visible.

The original 4B worked example shows why the intermediate choices matter: **MQ-Q5_K_1 used about 5.5% more space than the UD-Q5_K_XL recipe for about 32.6% lower measured KLD**, improving on the straight-line trade between neighboring baselines. Its [full results and tensor-group recipe breakdown](https://github.com/magiccodingman/MagicQuant/blob/main/wiki/overview.md#example) remain in the research overview.

### Take a discovered recipe set to another model

The [Qwen3.6-35B-A3B Uncensored release](https://huggingface.co/magiccodingman/Qwen3.6-35B-A3B-Uncensored-MagicQuant-GGUF) reuses **all ten configurations** from the original release on a related model by llmfan46, including the external-derived baseline. It rebuilds the artifacts and records fresh measurements rather than repeating the full discovery search.
Expand Down
2 changes: 2 additions & 0 deletions wiki/showcase.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,3 +67,5 @@ Evidence: [pinned method and model card](https://huggingface.co/magiccodingman/Q
The [original research overview](overview.md) retains the 4B survivor table, tensor-group recipe breakdown, nonlinear-trade example, methodology diagram, and context and GPU scheduling discussion. The project README draws on that material and links here for newer published results.

Its MQ-Q5_K_1 comparison uses the displayed values: `(2.88 / 2.73 - 1) × 100 ≈ 5.5%` additional storage and `(1 - 0.006632 / 0.009839) × 100 ≈ 32.6%` lower KLD. The [survivor-selection guide](docs/Nonlinear-Winners-And-Survivors.md) explains why a useful interior trade can deserve publication.

For the same example, linear interpolation at 2.88 GB between the UD-Q5_K_XL and LM-Q6_K anchors gives `0.009839 + (2.88 - 2.73) / (3.08 - 2.73) × (0.004640 - 0.009839) ≈ 0.007611`. The measured `0.006632` is about 12.9% below that reference line.
Loading