diff --git a/README.md b/README.md index 303aeea..308813e 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ [![NuGet version](https://img.shields.io/nuget/v/MagicQuant.svg)](https://www.nuget.org/packages/MagicQuant/) [![NuGet downloads](https://img.shields.io/nuget/dt/MagicQuant.svg)](https://www.nuget.org/packages/MagicQuant/) [![Build and tests](https://github.com/magiccodingman/MagicQuant/actions/workflows/dotnet.yml/badge.svg)](https://github.com/magiccodingman/MagicQuant/actions/workflows/dotnet.yml) -[![License: AGPL v3](https://img.shields.io/badge/License-AGPL_v3-blue.svg)](LICENSE) +[![License: AGPL v3](https://img.shields.io/badge/License-AGPL_v3-blue.svg)](https://github.com/magiccodingman/MagicQuant/blob/main/LICENSE) **Benchmark-driven GGUF quantization and mixed-precision hybrid discovery for llama.cpp.** @@ -18,7 +18,7 @@ It is a .NET command-line application that orchestrates llama.cpp and Python too 3. **Discover hybrids.** Use measured evidence and predictions to explore mixed-precision combinations with promising size/fidelity tradeoffs. 4. **Validate and select.** Measure candidates, reject poor or redundant trades, and export survivors with metadata and local provenance. -KLD and perplexity help evaluate fidelity; throughput and file size provide additional context. The results depend on the model, calibration/evaluation data, configuration, and hardware. A smaller KLD in one campaign is not a universal claim about downstream task quality. Read the [research overview](wiki/index.md) for the selection policy and its assumptions. +KLD and perplexity help evaluate fidelity; throughput and file size provide additional context. The results depend on the model, calibration/evaluation data, configuration, and hardware. A smaller KLD in one campaign is not a universal claim about downstream task quality. Read the [research overview](https://github.com/magiccodingman/MagicQuant/blob/main/wiki/index.md) for the selection policy and its assumptions. ## Support the project @@ -36,7 +36,7 @@ magicquant --version magicquant init-config --output config.yaml ``` -The package becomes available after the first successful release publication; until then use the [source installation instructions](docs/setup.md#build-from-source). +The package becomes available after the first successful release publication; until then use the [source installation instructions](https://github.com/magiccodingman/MagicQuant/blob/main/docs/setup.md#build-from-source). Edit the generated config for your source model, architecture identity, export destination, and storage. Initialize the external toolchain, validate the config, then start the campaign: @@ -46,7 +46,7 @@ magicquant pipeline --config ./config.yaml --check-config --strict-config magicquant pipeline --config ./config.yaml ``` -Initialization can download/build llama.cpp and install Python dependencies. NuGet installs MagicQuant, not model weights or a complete GPU toolchain. Follow the [installation guide](docs/setup.md) for native prerequisites, GPU setup, custom toolchains, and environment paths. +Initialization can download/build llama.cpp and install Python dependencies. NuGet installs MagicQuant, not model weights or a complete GPU toolchain. Follow the [installation guide](https://github.com/magiccodingman/MagicQuant/blob/main/docs/setup.md) for native prerequisites, GPU setup, custom toolchains, and environment paths. For updates: `dotnet tool update --global MagicQuant`. For reproducible runs, install a particular release with `--version X.Y.Z` and retain your config, model revision, and run provenance. @@ -69,30 +69,31 @@ learning: confirm_tensor_group_profile: true ``` -Custom YAML uses typed defaults for omitted values; it does not merge with the bundled tuning profile. Start with `init-config` when you want that complete profile. See [configuration](docs/configuration.md), [examples](examples), and the [command reference](docs/commands.md). +Custom YAML uses typed defaults for omitted values; it does not merge with the bundled tuning profile. Start with `init-config` when you want that complete profile. See [configuration](https://github.com/magiccodingman/MagicQuant/blob/main/docs/configuration.md), [examples](https://github.com/magiccodingman/MagicQuant/tree/main/examples), and the [command reference](https://github.com/magiccodingman/MagicQuant/blob/main/docs/commands.md). -**Plan scratch storage early.** Quantization writes and rereads large intermediate models, and storage can be a major throughput limitation. Fast SSD/NVMe scratch disks, especially separate physical devices, can materially improve throughput when IO is the bottleneck. Multiple folders on the same device still share its bandwidth. Allow space for concurrent intermediate artifacts and keep unrelated data out of managed scratch/export directories. See [storage](docs/storage.md) and [best practices](docs/best-practices.md). +**Plan scratch storage early.** Quantization writes and rereads large intermediate models, and storage can be a major throughput limitation. Fast SSD/NVMe scratch disks, especially separate physical devices, can materially improve throughput when IO is the bottleneck. Multiple folders on the same device still share its bandwidth. Allow space for concurrent intermediate artifacts and keep unrelated data out of managed scratch/export directories. See [storage](https://github.com/magiccodingman/MagicQuant/blob/main/docs/storage.md) and [best practices](https://github.com/magiccodingman/MagicQuant/blob/main/docs/best-practices.md). ## Learning from external quantizations External providers are optional. MagicQuant can run using its local baseline choices alone, but compatible external tensor assignments can provide valuable additional evidence. -**Unsloth is the maintainer's recommended starting point** for external GGUF baselines. MagicQuant can learn their tensor-group patterns, rebuild a controlled equivalent from your local source model, and benchmark it in your campaign. It does not simply trust an external file's label or score. Choose the exact matching model and revision, and review its license. See the [Unsloth configuration walkthrough](docs/best-practices.md#optional-unsloth-baselines) and [research explanation](wiki/docs/Learning-From-Existing-Quantizations.md). +**Unsloth is the maintainer's recommended starting point** for external GGUF baselines. MagicQuant can learn their tensor-group patterns, rebuild a controlled equivalent from your local source model, and benchmark it in your campaign. It does not simply trust an external file's label or score. Choose the exact matching model and revision, and review its license. See the [Unsloth configuration walkthrough](https://github.com/magiccodingman/MagicQuant/blob/main/docs/best-practices.md#optional-unsloth-baselines) and [research explanation](https://github.com/magiccodingman/MagicQuant/blob/main/wiki/docs/Learning-From-Existing-Quantizations.md). -For the same model, prefer linking to the original provider's surviving baselines. For a compatible variant they do not host, cloning can rebuild the full selected set locally. Learning tensor assignments does not automatically reproduce a provider's other processing techniques. See [publishing and cloning guidance](docs/best-practices.md#link-upstream-for-the-same-model-build-locally-for-variants). +For the same model, prefer linking to the original provider's surviving baselines. For a compatible variant they do not host, cloning can rebuild the full selected set locally. Learning tensor assignments does not automatically reproduce a provider's other processing techniques. See [publishing and cloning guidance](https://github.com/magiccodingman/MagicQuant/blob/main/docs/best-practices.md#link-upstream-for-the-same-model-build-locally-for-variants). ## Documentation | Start here | What you will find | | --- | --- | -| [Installation](docs/setup.md) | NuGet, native prerequisites, custom environments, source builds | -| [Configuration](docs/configuration.md) | YAML, overrides, read-only validation, profiles | -| [Commands](docs/commands.md) | Pipeline, setup, cloning, prediction validation | -| [Best practices](docs/best-practices.md) | Scratch disks, Unsloth, reproducibility, first campaigns | -| [Storage](docs/storage.md) | Persistent data, scratch leases, cache and output ownership | -| [Research](wiki/index.md) | Measurements, prediction, pruning, hybrid selection | -| [Contributing](CONTRIBUTING.md) | Development workflow, tests, code boundaries | -| [Releases](docs/releases.md) | Automatic versions and NuGet trusted publishing | +| [Installation](https://github.com/magiccodingman/MagicQuant/blob/main/docs/setup.md) | NuGet, native prerequisites, custom environments, source builds | +| [Configuration](https://github.com/magiccodingman/MagicQuant/blob/main/docs/configuration.md) | YAML, overrides, read-only validation, profiles | +| [Commands](https://github.com/magiccodingman/MagicQuant/blob/main/docs/commands.md) | Pipeline, setup, cloning, prediction validation | +| [Best practices](https://github.com/magiccodingman/MagicQuant/blob/main/docs/best-practices.md) | Scratch disks, Unsloth, reproducibility, first campaigns | +| [Manifests and cloning](https://github.com/magiccodingman/MagicQuant/blob/main/docs/manifests-and-cloning.md) | Publish a reproducible release; rebuild from another MagicQuant repository | +| [Storage](https://github.com/magiccodingman/MagicQuant/blob/main/docs/storage.md) | Persistent data, scratch leases, cache and output ownership | +| [Research](https://github.com/magiccodingman/MagicQuant/blob/main/wiki/index.md) | Measurements, prediction, pruning, hybrid selection | +| [Contributing](https://github.com/magiccodingman/MagicQuant/blob/main/CONTRIBUTING.md) | Development workflow, tests, code boundaries | +| [Releases](https://github.com/magiccodingman/MagicQuant/blob/main/docs/releases.md) | Automatic versions and NuGet trusted publishing | ## Development and history @@ -100,6 +101,6 @@ Application code lives in `src/`, tests in `tests/`, operational guides in `docs ## License -MagicQuant's original code and documentation are licensed under **GNU AGPL version 3 only** (`AGPL-3.0-only`). Commercial use is permitted subject to its terms. Distribution and remote interaction with modified versions carry source-availability obligations; the [license text](LICENSE) controls the details. +MagicQuant's original code and documentation are licensed under **GNU AGPL version 3 only** (`AGPL-3.0-only`). Commercial use is permitted subject to its terms. Distribution and remote interaction with modified versions carry source-availability obligations; the [license text](https://github.com/magiccodingman/MagicQuant/blob/main/LICENSE) controls the details. -This does not automatically relicense model weights or generated GGUFs. Model, dataset, external-provider, and third-party dependency licenses still apply. See [third-party notices](THIRD-PARTY-NOTICES.md). +This does not automatically relicense model weights or generated GGUFs. Model, dataset, external-provider, and third-party dependency licenses still apply. See [third-party notices](https://github.com/magiccodingman/MagicQuant/blob/main/THIRD-PARTY-NOTICES.md). diff --git a/assets/icon.png b/assets/icon.png index a2ca809..43c592e 100644 Binary files a/assets/icon.png and b/assets/icon.png differ diff --git a/docs/commands.md b/docs/commands.md index 22dd852..f4cdf01 100644 --- a/docs/commands.md +++ b/docs/commands.md @@ -23,6 +23,8 @@ The pipeline converts/loads the native source, reviews tensor groups, resolves i ## Clone known tensor configurations +For the complete publisher-to-user walkthrough, including manifest filenames and pointing at another Hugging Face release, see [manifests and cloning](manifests-and-cloning.md). + ```sh magicquant clone-repository-quants \ --config config.local.yaml \ diff --git a/docs/index.md b/docs/index.md index 63f04c9..8d27ed7 100644 --- a/docs/index.md +++ b/docs/index.md @@ -4,6 +4,7 @@ Start with the [project briefing and quick start](../README.md). These guides de - [Setup](setup.md): NuGet installation, toolchain preparation, source builds, troubleshooting. - [Configuration](configuration.md): YAML profile, CLI overrides and path rules. +- [Manifests and cloning](manifests-and-cloning.md): release upload layout, JSON evidence and cloning from other repositories. - [Commands](commands.md): discovery, clone/export and prediction validation. - [Best practices](best-practices.md): scratch disks, optional Unsloth baselines and reproducibility. - [Storage](storage.md): runtime/model directories, caches, scratch and provenance. diff --git a/docs/manifests-and-cloning.md b/docs/manifests-and-cloning.md new file mode 100644 index 0000000..fc91fa8 --- /dev/null +++ b/docs/manifests-and-cloning.md @@ -0,0 +1,126 @@ +# Publish manifests and clone another MagicQuant release + +A MagicQuant release can carry its tensor configurations alongside its GGUF downloads. That lets someone else point the CLI at the release and rebuild those configurations from their own compatible source model—for example, a fine-tuned variant that the original publisher does not host. + +There are two different kinds of config: + +- **Your YAML campaign config** selects local paths, hardware/storage settings, architecture identity, imatrix inputs, and runtime policies. +- **The clone JSON manifest** describes the source release's selected artifacts and their per-tensor quantization assignments. It is generated by MagicQuant, not a replacement for your YAML. + +## For publishers: what to upload + +Use the generated export directory as the release layout. For a Hugging Face **model repository**, keep the README and locally exported GGUFs at the repository root and retain the manifest directory and filenames: + +```text +YOUR-HF-OWNER/YOUR-MODEL-MagicQuant/ +├── README.md +├── .gguf +├── +└── magicquant-manifest/ + ├── magicquant.clone-configs.json + ├── magicquant.final-survivors.json + ├── magicquant.hybrid-map.json + ├── magicquant.replacements.json + ├── magicquant.isolation-samples.json + └── magicquant.bad-trades.json +``` + +The files present vary by workflow and available evidence. **`magicquant-manifest/magicquant.clone-configs.json` is the required input for cloning.** The other JSON files explain the results and should be retained when generated. A README, GGUF, or final-survivor report alone is not a clone manifest. + +Upload the export contents at the repository root, not inside an extra `Final_Outputs/` directory. Preserve generated filenames and relative locations so model-card downloads and manifest lookup work. Review the model card's license, base-model attribution, provider credits, and any source paths before publishing; the application's AGPL license is not automatically the model's license. + +Publish the intended release outputs, not your entire working directory. Local `Runs/` provenance, databases, `CloneSource/`, scratch, logs, and private YAML are not required by someone cloning your release. Some JSON source/provenance fields can contain paths or URLs; review those too. MagicQuant generates export files locally; the CLI does not upload them to Hugging Face for you. + +When an external provider already hosts the same model, leave pipeline `output.export_external_learned_baselines: false` and retain upstream links instead of duplicating their downloads. The clone manifest can still contain the external survivor's tensor assignments, recovered from learned evidence even without a local exported copy. See [provider attribution and model variants](best-practices.md#link-upstream-for-the-same-model-build-locally-for-variants). + +## What each manifest contains + +| File under `magicquant-manifest/` | Purpose | +| --- | --- | +| `magicquant.clone-configs.json` | Selected artifact names, base quantization, exact tensor-name → quantization-type maps, source identity and source metrics; this is the rebuild input. | +| `magicquant.final-survivors.json` | Final selection metrics, sizes, artifact details and replacement lineage from discovery. | +| `magicquant.hybrid-map.json` | Tensor-group assignments and effective-state details for hybrid artifacts. | +| `magicquant.replacements.json` | Which candidates/baselines were replaced and why. | +| `magicquant.isolation-samples.json` | Measured isolated changes used as discovery evidence. | +| `magicquant.bad-trades.json` | Recorded poor-trade/pruning decisions. | +| `magicquant.clone-benchmarks.json` | New measurements from a clone run, including source metrics for comparison. | + +The clone schema is defined by `MagicQuantCloneManifest` and `MagicQuantCloneArtifact` in `src/MagicQuant/Models/RepositoryCloneModels.cs`. Generated JSON uses fields such as `SchemaVersion`, `Artifacts`, `FileName`, `BaseQuant`, `UsedImatrix`, `SourceKld`, and `TensorTypes`. `TensorTypes` is the actual reconstruction payload; a friendly quant label is not a substitute for the map. Keep the generated map intact rather than hand-authoring a few representative tensors. + +## For users: point at another MagicQuant repository + +Install MagicQuant and prepare its toolchain as described in [setup](setup.md). Obtain the **complete local source model** you want to quantize; an existing GGUF alone is not that input. Create an editable YAML: + +```sh +magicquant init-config --output clone.yaml +``` + +Edit its model directory, architecture identity, output directory, scratch disks, and any required imatrix settings. Then substitute a real publisher's Hugging Face model repository ID in both commands below: + +```sh +magicquant clone-repository-quants \ + --config ./clone.yaml \ + --source-repo YOUR-HF-OWNER/YOUR-MODEL-MagicQuant \ + --check-config --strict-config + +magicquant clone-repository-quants \ + --config ./clone.yaml \ + --source-repo YOUR-HF-OWNER/YOUR-MODEL-MagicQuant +``` + +`--source-repo` takes an **`owner/repository` Hugging Face model ID**, not the MagicQuant GitHub code repository URL and not a local Git checkout. It looks for `magicquant-manifest/magicquant.clone-configs.json`, with a legacy fallback to `magicquant.clone-configs.json` at the repository root. It also attempts to retrieve the known optional evidence files. + +The read-only check validates local config/input paths and the source option selection. It does **not** download or validate the remote manifest; the real clone run does that. Private/gated sources need the appropriate access in the configured Hugging Face environment. A public source is the simplest sharing route. + +Clone mode rebuilds every artifact entry present in that manifest, including external-derived entries, and benchmarks the results locally. It does not require locally re-hosted external GGUFs in the source release. You do not need to copy the source author's provider configuration just to reproduce the manifest maps. If you deliberately add external repositories to your own YAML, those are additional configured runtime inputs. + +**`output.export_external_learned_baselines` is a pipeline export choice, not a clone filter.** The clone command already builds the external-derived manifest entries locally, which is appropriate when your variant is not hosted by their original providers. + +## Local manifests and pinned URLs + +Use exactly one of `--source-repo` or `--source-json`. + +For an already downloaded manifest: + +```sh +magicquant clone-repository-quants \ + --config ./clone.yaml \ + --source-json /data/releases/source/magicquant-manifest/magicquant.clone-configs.json +``` + +`--source-json` also accepts a direct HTTP(S) URL returning the JSON file. It must be a raw/download URL, not an HTML file-view page. For a fixed Hugging Face revision, replace `COMMIT_SHA` with the source repository's actual commit: + +```sh +magicquant clone-repository-quants \ + --config ./clone.yaml \ + --source-json 'https://huggingface.co/YOUR-HF-OWNER/YOUR-MODEL-MagicQuant/resolve/COMMIT_SHA/magicquant-manifest/magicquant.clone-configs.json' +``` + +`--source-repo` currently uses the repository's default revision; there is no clone `--source-revision` flag. Use a pinned raw URL or a retained local JSON file when you need a stable reconstruction input. Direct URL loading does not automatically fetch neighboring evidence files or reuse authenticated Hub downloads. For private sources, downloading the JSON yourself and passing a local path is an alternative. A single-file input is sufficient for reconstruction, but do not assume it brings the source release's entire evidence archive with it. + +## Compatibility and limits + +By default, the manifest tensor names must match the target model's tensor inventory. An architecture name or similar model size alone does not prove compatibility. A related fine-tune can be a useful cloning target, but weight changes can alter which configurations work well. + +`--allow-missing-manifest-tensors` permits a manifest that covers a strict subset of target tensors; uncovered target tensors use the base quantization. `--missing-manifest-base-quant Q8_0` selects a fallback base quantization for that supported path. These options are explicit compatibility choices, not general fixes for mismatched architectures. Manifest-only tensor names that do not exist in the target remain a mismatch. + +A manifest does not contain source weights, a frozen software environment, or an imatrix payload. `UsedImatrix` records source context; inspect and configure the target's imatrix requirements rather than assuming the JSON supplies calibration data. Unsupported tensor quantization types or converter/model combinations can still fail. + +MagicQuant copies tensor quantization assignments. It does not automatically reproduce a provider's other weight transformations, calibration recipes, or custom processing. Clone mode measures the rebuilt artifacts but does not rerun full hybrid discovery, inherit proof of optimality, or compare against the original provider's exact artifact. See [limits of tensor-configuration copying](best-practices.md#limits-of-tensor-configuration-copying). + +## Publishing a clone of a clone + +The clone output includes rebuilt GGUFs, a generated README with a clone notice, the clone configuration manifest, and `magicquant.clone-benchmarks.json`. When using `--source-repo`, available source evidence is archived as well. Source discovery reports remain **source-model evidence**; the clone benchmark summary contains the new model's measurements. Do not present archived source reports as a fresh discovery campaign on your variant. + +You can upload that new export layout to your own Hugging Face model repository. Another user can then point `--source-repo` at your repository in the same way. Preserve attribution, the clone notice, and the manifest folder so the next user can understand both the origin and the limits of the result. + +## Troubleshooting + +| Symptom | Check | +| --- | --- | +| Manifest cannot be downloaded | Correct HF model ID, repository access, exact manifest filename and folder; avoid nesting the release under another directory. | +| JSON parse failure | Use the actual clone JSON, not a model-card page, GGUF, or another report. | +| Zero artifacts or empty `TensorTypes` | Publish the completed generated clone manifest, not a placeholder or partial map. | +| Tensor mismatch | Verify source/target model compatibility before considering subset overrides. | +| Missing imatrix or unsupported quant type | Check target calibration settings and the installed llama.cpp version. | +| Unexpected copied source metrics | Compare the fresh `magicquant.clone-benchmarks.json`; archived discovery reports belong to the source. | diff --git a/docs/nuget-readme.md b/docs/nuget-readme.md deleted file mode 100644 index efdf4a3..0000000 --- a/docs/nuget-readme.md +++ /dev/null @@ -1,34 +0,0 @@ -# MagicQuant - -**Benchmark-driven GGUF quantization and mixed-precision hybrid discovery for llama.cpp.** - -MagicQuant measures baselines, learns tensor-group assignments, discovers promising hybrid quantizations, and validates size/fidelity tradeoffs before exporting selected GGUF artifacts. It is a command-line tool, not an evolutionary search algorithm or a new quantization format. - -## Install - -Install the .NET 10 SDK, then: - -```sh -dotnet tool install --global MagicQuant -magicquant init-config --output config.yaml -``` - -Edit model, architecture, output, and scratch paths, then prepare the native toolchain and run: - -```sh -magicquant initialize-llama-cpp -magicquant pipeline --config config.yaml --check-config --strict-config -magicquant pipeline --config config.yaml -``` - -NuGet does not bundle model weights or a ready-to-use GPU toolchain. Linux is the tested campaign platform; Windows has automated build/unit/package checks but full campaigns remain unvalidated. - -Fast, separate physical scratch disks can help substantially when repeated large GGUF writes are the bottleneck. External quantization providers are optional; Unsloth is the maintainer's recommended starting point for compatible tensor-assignment evidence. - -- [Project briefing and research](https://github.com/magiccodingman/MagicQuant) -- [Installation and prerequisites](https://github.com/magiccodingman/MagicQuant/blob/main/docs/setup.md) -- [Configuration](https://github.com/magiccodingman/MagicQuant/blob/main/docs/configuration.md) -- [Scratch disks and optional Unsloth learning](https://github.com/magiccodingman/MagicQuant/blob/main/docs/best-practices.md) -- [Support spare-time development and storage costs](https://sayou.biz/support) - -MagicQuant is licensed under **AGPL-3.0-only**. Model weights and generated GGUFs retain their applicable licenses. See [license and third-party notices](https://github.com/magiccodingman/MagicQuant/blob/main/THIRD-PARTY-NOTICES.md). diff --git a/docs/releases.md b/docs/releases.md index b94b364..3c75230 100644 --- a/docs/releases.md +++ b/docs/releases.md @@ -1,6 +1,6 @@ # NuGet releases -MagicQuant is packaged as a .NET tool, package ID `MagicQuant`, executable `magicquant`. `.github/workflows/publish-nuget.yml` publishes when a commit reaches `release`, normally through a merged PR. Pushes directly to `release` also trigger it; use branch protection to require PRs if desired. Manual dispatch is available to retry publication and only runs on `release`. +MagicQuant is packaged as a .NET tool, package ID `MagicQuant`, executable `magicquant`. The package embeds the root `README.md` and `assets/icon.png`; package checks verify both files byte-for-byte and validate their NuGet metadata. Keep root README links absolute so the same content works on GitHub and NuGet. `.github/workflows/publish-nuget.yml` publishes when a commit reaches `release`, normally through a merged PR. Pushes directly to `release` also trigger it; use branch protection to require PRs if desired. Manual dispatch is available to retry publication and only runs on `release`. ## Trusted publishing setup diff --git a/docs/storage.md b/docs/storage.md index bfe81f1..772a659 100644 --- a/docs/storage.md +++ b/docs/storage.md @@ -23,7 +23,7 @@ The runtime root and model work directory are different things. With default set PredictionValidation/ # prediction report default ``` -Output is configurable and only directories needed by a run are created. Other service-specific files may also appear. Final exports put JSON evidence under `magicquant-manifest/`, including clone configurations, final survivors, replacements, hybrid maps, isolation samples, and bad-trade reports as appropriate to the workflow. `MagicQuantManifestPathService` owns those filenames and links. +Output is configurable and only directories needed by a run are created. Other service-specific files may also appear. Final exports put JSON evidence under `magicquant-manifest/`, including clone configurations, final survivors, replacements, hybrid maps, isolation samples, and bad-trade reports as appropriate to the workflow. `MagicQuantManifestPathService` owns those filenames and links. See [manifests and cloning](manifests-and-cloning.md) for each file's role and the layout to upload. ## Durable truth diff --git a/scripts/package_smoke.py b/scripts/package_smoke.py index fcca021..3c6d08b 100644 --- a/scripts/package_smoke.py +++ b/scripts/package_smoke.py @@ -32,11 +32,16 @@ def main(): for native in ["libe_sqlite3.so", "e_sqlite3.dll", "libduckdb.so", "duckdb.dll"]: assert any(n.endswith('/'+native) for n in names), f"Missing native asset: {native}" assert any(n.startswith("licenses/") for n in names), "Missing third-party license texts" + repository = Path(__file__).resolve().parent.parent + assert archive.read("README.md") == (repository / "README.md").read_bytes(), "Package README differs from the root README" + assert archive.read("icon.png") == (repository / "assets/icon.png").read_bytes(), "Package icon differs from the repository icon" root = ET.fromstring(archive.read("MagicQuant.nuspec")) ns = {"n": root.tag.split("}")[0][1:]} meta = root.find("n:metadata", ns) version = meta.find("n:version", ns).text assert meta.find("n:license", ns).text == "AGPL-3.0-only" + assert meta.find("n:readme", ns).text == "README.md" + assert meta.find("n:icon", ns).text == "icon.png" with tempfile.TemporaryDirectory(prefix="magicquant package smoke ") as temp: root = Path(temp) feed = root / "feed" diff --git a/scripts/test_doc_links.py b/scripts/test_doc_links.py index be886b5..b7e09e9 100644 --- a/scripts/test_doc_links.py +++ b/scripts/test_doc_links.py @@ -15,6 +15,14 @@ def test_local_documentation_targets_exist(self): for path in files: for match in re.finditer(r'\]\(([^)]+)\)', path.read_text(encoding='utf-8')): url = match.group(1).split(' "')[0].strip('<>') + # The shared root README uses absolute canonical URLs so its + # links also work on NuGet. Still validate their local targets. + for kind in ('blob', 'tree'): + prefix = f'https://github.com/magiccodingman/MagicQuant/{kind}/main/' + if url.startswith(prefix): + target = unquote(url[len(prefix):].split('#')[0]) + if not (ROOT / target).exists(): + errors.append(f'{path.relative_to(ROOT)}: {url}') if url.startswith(('https:', 'http:', 'mailto:', '#')): continue target = unquote(url.split('#')[0]) diff --git a/src/MagicQuant/MagicQuant.csproj b/src/MagicQuant/MagicQuant.csproj index 3a5621d..3d684c0 100644 --- a/src/MagicQuant/MagicQuant.csproj +++ b/src/MagicQuant/MagicQuant.csproj @@ -46,7 +46,7 @@ - +