feat: private crate registry via JFrog Artifactory (#12) - #684
Merged
Conversation
Eighteen releases in, `publish = false` and no crates.io presence meant
no colleague could depend on this crate. This wires it to Lab271's
Artifactory tenant as a private Cargo registry, opt-in.
Cargo has no per-project registry config that lives outside the working
tree — there is no `jf cargo-config` and no `jf cargo` command at all,
so `.cargo/config.toml` and `CARGO_*` env vars are the only mechanisms
on offer. That matters here because this repository is public and the
Artifactory Cargo index is not anonymously readable:
$ curl -o /dev/null -w '%{http_code}' \
.../api/cargo/lab-cargo-dev/index/config.json
401
The index also advertises `"auth-required": true`, which Cargo honours
for crate downloads as well as index reads. So a committed
`.cargo/config.toml` carrying a source replacement would 401 every
anonymous `cargo build` — every outside contributor, every fork — and
would do it at dependency-fetch time, with an authentication error that
gives the contributor no hint that the fix is to delete a file they did
not add. The npm pilot of the same evaluation rejected a committed
`.npmrc` for exactly this reason.
Hence: commit `.cargo/config.toml.example`, gitignore
`.cargo/config.toml`. The default clone is untouched and resolves from
crates.io as before. ADR-0038 records the decision and the alternatives.
`package.publish` stays `false`, deliberately. It does not need to
change: `cargo package` works under it, and the resulting `.crate`
deploys with `jf rt upload`, which Artifactory indexes into a valid
registry entry on its own (~5s later, with deps/features/cksum parsed
from Cargo.toml). Setting `publish = ["lab-cargo-dev"]` would also work
and would *not* open a path to crates.io — `cargo publish --registry
crates-io` still refuses — but it is unnecessary, and whether this crate
should be publishable at all is a separate decision that is not this
one's to make.
Two things the docs call out because they are not obvious:
- `sqlite-rs` is already taken on crates.io by an unrelated crate, 19
versions up to 0.3.7. `lab-cargo-dev` merges its crates.io cache with
our local repo, so one index path serves 20 versions from two
projects with nothing distinguishing them. Ours resolves correctly
today only because the ranges do not overlap. `lab-cargo-prod` has no
remote and serves 0.18.10 alone.
- Consumers should depend on us via the named-registry form, not the
source replacement. Under source replacement the consumer's lockfile
records our private crate as coming from crates.io, which is untrue
and makes deny.toml's `sources` check pass it silently.
The workflow skips cleanly whenever JF_ACCESS_TOKEN is absent, which is
always the case on forked PRs — a JFrog-caused red build would count
against the evaluation's reliability criterion, whose threshold is zero.
Refs: Lab271/labs-jfrog-poc#12
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`workflow_dispatch` only fires for a workflow that already exists on the default branch, so from a feature branch there is no way to make the publish and promote jobs actually run. An unexecuted workflow is not evidence that it works, and the promote job carries the evaluation's hard-fail assertion (promoted digest identical to the dev digest), so it needs a real run rather than a reading. Reverted in the next commit, before merge. Also makes the consume step skip when the current version is not in lab-cargo-prod yet, which is the normal state on a PR that bumps the version. Failing there would be a JFrog-caused CI failure, and the evaluation's threshold for those is zero. Refs: Lab271/labs-jfrog-poc#12 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Artifactory's Cargo index advertises `"auth-required": true`, and Cargo then refuses to query it at all unless a credential provider is named: "authenticated registries require a credential-provider to be configured". `CARGO_REGISTRIES_*_TOKEN` alone is not enough — the provider has to be listed before the env var is consulted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Found the hard way, and it broke a real consumer:
* `cargo package` is not byte-reproducible across machines. Commit
e04330e produced sha256:eb1deeae… on a laptop and sha256:84074b9f…
on a runner.
* `lab-cargo-dev-local` and `lab-cargo-prod-local` are mutable.
Re-uploading 0.18.10 overwrote the bytes silently — no error, no
warning — including in prod, where that version already carried
status `Released`.
* Cargo pins checksums in Cargo.lock, so that is not a harmless
re-upload. A consumer pinned to the old digest now fails outright:
"checksum for `sqlite-rs v0.18.10` changed between lock files ...
unable to verify that `sqlite-rs v0.18.10` is the same as when the
lockfile was generated". Reproduced, not theorised.
Publish and promote now skip cleanly when the version already exists,
rather than clobbering it. Bumping the version is the way to release new
bytes.
Repository immutability on the prod local repo
(Lab271/labs-jfrog-poc#4) is the actual fix and needs platform admin we
do not have. Until then this guard is the only thing between a re-run
and a broken downstream lockfile.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The publish and promote jobs have now run for real on this branch (run 33790369579, all four jobs green, "Promoted bytes identical to dev — no rebuild in the path"), and the overwrite guard has been verified skipping cleanly on a re-run (run 33790771562). The trigger has served its purpose. Back to tags and workflow_dispatch only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the Cargo half of Lab271/labs-jfrog-poc#12.
Eighteen releases in,
publish = falseand no crates.io presence meant no colleague could depend on this crate. This wires it to Lab271's Artifactory tenant as a private Cargo registry — opt-in, so a default clone is entirely unaffected.The public-repo trade-off, and why the config is not committed
Cargo has no per-project registry config that lives outside the working tree. npm has
.npmrc, and the JFrog CLI can sidestep even that withjf npm-config. There is nojf cargo-configand nojf cargocommand at all —jf --helpsays so outright. So.cargo/config.tomlandCARGO_*env vars are the only mechanisms on offer.That matters because this repository is public and the Artifactory Cargo index is not anonymously readable:
The index also advertises
"auth-required": true, which Cargo honours for crate downloads as well as index reads. A committed source replacement would therefore 401 every anonymouscargo build— every outside contributor, every fork — at dependency-fetch time, with an authentication error that gives the contributor no hint that the fix is to delete a file they did not add.So:
.cargo/config.toml.exampleis committed,.cargo/config.tomlis gitignored. ADR-0038 records the decision and the four alternatives rejected.package.publishstaysfalseIt does not need to change.
cargo packageworks under it, and the resulting.cratedeploys withjf rt upload, which Artifactory indexes into a valid registry entry on its own (~5s later, withdeps,featuresandcksumparsed fromCargo.toml).publish = ["lab-cargo-dev"]would also work, and is narrower than it looks —cargo publish --registry crates-iostill refuses with "The registrycrates-iois not listed in thepackage.publishvalue", and a barecargo publishauto-targets the single allowed registry. But it is unnecessary, and only the upload path produces build-info. Whether this crate should be publishable at all is a separate decision and not this PR's to make.Two things worth knowing before relying on this
sqlite-rsis already taken on crates.io by an unrelated crate — 19 versions, up to0.3.7.lab-cargo-devmerges its crates.io cache with our local repo, so one index path serves 20 versions from two different projects with nothing distinguishing them. Ours resolves correctly today only because the version ranges do not overlap.lab-cargo-prodhas no remote and serves0.18.10alone. Renaming the crate is the real fix, before any public release.Cargo.lockrecords our private crate assource = "registry+https://github.com/rust-lang/crates.io-index"— untrue, and it makesdeny.toml'ssourcescheck (unknown-registry = "deny",allow-registry= crates.io only) pass a private-registry dependency silently.Test plan
Verified against the live tenant, not reasoned about:
cargo fetch --lockedthroughlab-cargo-devCargo.lockunchanged — all 116sourceentries still point at crates.iolab-cargo-dev-local/crates/sqlite-rs/sqlite-rs-0.18.10.crate,sha256:eb1deeae…e04330e8in oneGET /api/build/sqlite-rs/2?project=lab--copy=truelab-cargo-prod-local, digestsha256:eb1deeae…— identical, no rebuildsqlite-rs = "0.18", resolved0.18.10from the registry, compiled and linked it, and ranAnonymous access was confirmed broken before choosing the mechanism (the 401 above), which is the whole basis for ADR-0038.
Notes for a human
jfrog-prodenvironment does not exist yet. GitHub creates it on this workflow's first run with no protection rules, so the promote job is currently ungated. Add required reviewers under Settings → Environments →jfrog-prodbefore treating it as an approval gate. Until then the environment records who triggered the run, not who approved it.JF_ACCESS_TOKENis a repo secret carrying project-admin rights onlab, so any workflow in this repository can promote to prod, and one credential covers both build and promote. That is a least-privilege regression against the OIDC design in labs-jfrog-poc#6 and is recorded there, not fixed here.pushtrigger for this branch —workflow_dispatchonly fires for a workflow that already exists on the default branch, so there was no other way to make the publish and promote jobs actually run. All four jobs went green in run 33790369579 (Promoted bytes identical to dev — no rebuild in the path), and the overwrite guard was verified skipping cleanly in run 33790771562. The trigger is gone from the final state;on:is back to tags andworkflow_dispatch.[registry] global-credential-providers = ["cargo:token"]named explicitly beforeCARGO_REGISTRIES_*_TOKENis consulted at all (not mentioned in Artifactory's Set Me Up), and re-publishing a version silently overwrites released bytes — see the third commit, which is the one worth reading.Xray
Reported in full in the PoC findings, but the short version, since it bears on this repo's existing supply-chain gate:
jf auditdoes not recognise a Rust project. It reports the tree as[unknown]and generates an SBOM with no library components, so the 116-crate closure thatmake check-denyandsqlite-rs-dev.cdx.jsonalready cover is never examined. Xray does have Cargo CVE data and applies it to a.cratescanned withjf scan— confirmed against a deliberately vulnerabletime 0.1.44, which reportsCVE-2020-26235, typecargo— but this crate has zero runtime dependencies, so there is nothing there to find.make check-denyremains the supply-chain gate. Artifactory adds distribution, not assurance. No change todeny.toml, the SBOMs, or any existing gate is proposed here.Spend: matched estimate.
🤖 Generated with Claude Code