feat(upload, manifest): report the uploaded file's SHA-256 and record it on manifest entries (V2-1564)

[Nic-dorman]

Uploads now report the uploaded file's SHA-256, and ADR-0006 manifest entries can record it. Extraction checks every download against it.

What

Uploads

• The read loop that feeds stream_encrypt hashes each 8 KiB buffer. The SHA-256 comes back with the DataMap from the encryption task, so it costs no extra pass over the file.
• New fields:
  • FileUploadResult::sha256 and PreparedUpload::sha256, both #[non_exhaustive], so these are additive. The value is carried through every finalize path, including the resumable wave and merkle handles;
  • DataUploadResult::sha256 for in-memory data.

Manifests (ADR-0006 amendment; the ADR owner approved the field)

• Format: ManifestEntry::sha256, an optional 32-byte bin written between size and source. Under the format's forward-compatibility rules it is optional within version 1:
  • readers that predate it skip it (the structure check and serde both ignore unknown keys);
  • entries without it encode byte-for-byte as before (pinned by a new golden fixture).
• Builder: ManifestBuilder records it for every file it uploads. Entries added by address, or with a DataMap fetched from the network, have none: the creator never read their bytes.
• Extraction: each downloaded temporary file that has a recorded hash is hashed before it takes its name. A mismatch fails that entry (ManifestError::Sha256Mismatch) and leaves nothing at the target.
• Browser: decodeManifest / parseManifestLink entries expose sha256 as hex.

CLI

• ant file upload prints SHA-256:, --json adds sha256, and the hash is recorded in the upload history.
• ant manifest show prints each recorded hash under its entry and adds it to --json.

ADR-0006 is amended in the data model, normative schema, creation, history, extraction, surface, consequences and validation sections. It also gets the daemon-API wording its owner confirmed: no daemon API is part of the initial release. WithAutonomi/ant-sdk adds one under V2-1570.

Why

The DataMap's hashes are per chunk, and the content address proves integrity only inside Autonomi. Fixity (V2-440), signed receipts (V2-454), Indelible's end-to-end ingest check (V2-1427) and outside tools all expect a standard whole-file hash. Computing it in the core means every client gets it: CLI, antd and the SDKs.

Linear issue

Closes V2-1564

Risk tier

• T0 — docs / tooling / CI / pure UX-output. Repo CI only.
• T1 — client-only, no network-facing behavior change. CI + prod compat smoke.
• T2 — node/client logic with behavioral surface, no protocol/format/economics change. Dev testnet + ADR.
• T3 — protocol / storage format / payments / routing. T2 evidence + adversarial testing.

Nothing on the wire or on the network changes. The manifest is a client-side file format that is never stored on the network, and the new field is optional within its version.

Compatibility

• Wire: none.

• Storage: none on the network. The .ant format gains an optional entry field within version 1:

  • manifests with it are read by existing clients, which ignore the field;
  • manifests without it are unchanged.

• API: additive fields on FileUploadResult and PreparedUpload (both #[non_exhaustive]). Three changes are breaking for consumers that use struct literals or exhaustive patterns:

  • a new public field on DataUploadResult;
  • a new public field on ManifestEntry;
  • a new ManifestError::Sha256Mismatch variant.

  CLI output gains a line in ant file upload and ant manifest show, and a sha256 key in both --json outputs.

Semver impact

• breaking
• feature
• fix

It's breaking for ant-core Rust consumers only via struct literals or exhaustive matches, as listed above. On the 0.x line this is a minor bump.

Test evidence

• Unit tests (ant-core):
  • the read loop's hash equals a one-shot digest, over a 4-chunk file that spans many reads;
  • a golden fixture pins the encoded position of sha256;
  • entries without it encode unchanged;
  • a hash of the wrong length is rejected;
  • extraction's file hash matches a one-shot digest;
  • the finalize assemble tests carry it through both complete and resumable outcomes.
• Suites:
  • cargo test -p ant-core --lib: 830/834 pass. The other 4 are node::daemon::forward::runner timing tests, which this PR doesn't touch; they pass 11/11 when run on their own, both parallel and single-threaded.
  • cargo test -p ant-cli: all pass.
• E2E on the in-process local network with Anvil:
  • e2e_manifest (7/7), including the new uploads_record_sha256_and_extraction_checks_it. A built manifest records the right hash for every uploaded file, and it extracts. A tampered hash fails only that entry, leaving no file and no temporary file behind;
  • e2e_file: the direct upload, and the external-signer prepare → finalize_upload path, both report the right hash.
• Checks:
  • cargo clippy --all-targets --all-features -- -D warnings (native) and the wasm32-unknown-unknown browser-wasm clippy: clean;
  • cargo fmt --check: clean;
  • scripts/adr-governance.py: passed.
• Downstream: WithAutonomi/ant-sdk V2-1570 (antd manifest endpoints) builds against this branch for the antd side of V2-1564.

New dependency

sha2 0.10, as a direct dependency of ant-core. It was already in the lockfile transitively, so no new crate enters the graph.

ADR

ADR-0006 (Proposed) is amended in this PR: docs/adr/ADR-0006-file-manifests-and-links.md. Not required for T1.

Mitigation / rollback

Revert the PR. Manifests written with sha256 stay readable by every client, because the field is optional and ignored by readers that don't know it.

🤖 Generated with Claude Code

[dirvine]

Review of 2eace7374d0642376e799458a4fb9e63c2b7cc4c

No severe defect introduced by this PR was found in the completed review work. This result is provisional, not a formal approval.

Code checks

• The native uploader hashes the plaintext in the same read loop that feeds encryption. Read/encryption failures do not publish a successful hash/DataMap result.
• The digest is carried through prepared uploads, payment/finalisation and resume results, then exposed by the CLI and browser API.
• The manifest field is optional. Missing hashes retain the old version-1 encoding. Existing readers can skip the new field.
• Extraction checks the downloaded temporary file before rename. A mismatch fails that entry and removes the temporary file, rather than publishing unverified content.

Verification

I ran these commands locally at the reviewed head; each returned exit code 0:

• cargo test -p ant-core --locked --lib sha256 — 5 passed.
• cargo test -p ant-core --locked --lib data::manifest:: — 54 passed.
• cargo test -p ant-core --locked --lib assemble_ — 5 passed.
• cargo test -p ant-cli --locked — 31 passed.
• cargo check -p ant-core --locked --target wasm32-unknown-unknown --no-default-features --features browser-wasm — passed. This is a build check, not browser runtime execution.

Exact-head CI reports native unit, E2E and Merkle E2E success on Linux/macOS, plus Browser WASM and real Chromium/local-node success. I inspected the new hash-mismatch E2E test, but did not rerun network E2E locally.

Separate CI failure

Security Audit is still failing. It reports rustls 0.23.45 advisories RUSTSEC-2026-0346, -0347, -0348 and -0349. That version is also present in the base commit 225a6768589ecbb6a0c77d70df329f343b22776f; this PR's only lockfile change adds the already-locked sha2 dependency to ant-core. This is an inherited dependency issue, not a new SHA-256 implementation defect. It remains unresolved; CI is not fully green. The earlier PR-template failure has a successful later run.

Independent review participation

Two distinct rounds were requested from GLM 5.3/OpenRouter (z-ai/glm-5.3), local Qwen 3.8 (qwen38-27b) and local DS4 (deepseek-v4-flash):

• Correctness: GLM and Qwen completed with no blocker; DS4 timed out.
• Adversarial security/lifecycle: Qwen completed with no blocker; GLM returned an incomplete termination; DS4 timed out.

3/6 role reviews completed, from 2/3 named models. The completed opinions agree; there is no substantive dissent in those opinions. Missing reviews are not agreement. Full team consensus and the full review gate are not established.

Before formal approval, complete the missing review roles. The inherited Security Audit failure also needs resolution or a separate authorised decision. No approval or merge action is taken by this comment.

[dirvine]

APPROVE — reviewed head 2eace7374d0642376e799458a4fb9e63c2b7cc4c

No severe defect introduced by this PR was found in the completed code checks, tests or independent reviews. The live PR head and base are unchanged from the reviewed revisions.

Authority and scope: Benjamin Pierce explicitly authorised Hermes in the review thread to submit a formal verdict on this PR based on the reviews that completed. This supersedes the earlier hold on submitting a formal verdict for this PR only. It does not mean the full review panel completed, does not waive CI requirements, and does not authorise a merge or release.

Evidence supporting this verdict

• The native uploader hashes the plaintext in the same read loop that feeds encryption. Read/encryption errors do not publish a successful hash/DataMap result.
• The digest passes through prepared uploads, payment/finalisation and resume results to the CLI and browser API.
• The manifest hash is optional and preserves the old version-1 encoding when absent. Extraction verifies the temporary file before publishing it; a mismatch fails the entry and removes the temporary file.
• Previously executed local checks at this unchanged head all returned exit code 0: cargo test -p ant-core --locked --lib sha256, cargo test -p ant-core --locked --lib data::manifest::, cargo test -p ant-core --locked --lib assemble_, cargo test -p ant-cli --locked, and the browser-wasm target build check. The latter is a build check, not local browser runtime execution.
• Current exact-head CI reports successful native unit, E2E and Merkle E2E runs on Linux/macOS, Browser WASM, and real Chromium/local-node tests. Network E2E was not rerun locally. The earlier PR-template failure has a later successful run.

Review limits and unresolved CI

3/6 whole-role reviews completed, from 2/3 named models. The panel assessment remains provisional; full team consensus is not established.

• GLM 5.3/OpenRouter (z-ai/glm-5.3): correctness review completed with APPROVE; adversarial extraction-scope retry also completed with APPROVE, but the upload scope did not complete. The scoped retry does not complete the whole adversarial role.
• Qwen 3.8 (qwen38-27b): correctness and adversarial reviews completed with APPROVE.
• DS4 Flash (deepseek-v4-flash): no substantive completed review; requests timed out, including retries.

No material dissent or introduced blocker was reported by the completed reviews. Missing responses are not agreement.

Security Audit still fails on rustls 0.23.45 advisories RUSTSEC-2026-0346, -0347, -0348 and -0349. The same version is present in base 225a6768589ecbb6a0c77d70df329f343b22776f; this PR's lockfile delta only adds the already-locked sha2 dependency to ant-core. This is an inherited dependency issue, not an introduced hash implementation defect. This approval does not resolve or waive that failure, or declare the PR ready to merge.

Submitted by Hermes through the configured GitHub account, under Benjamin Pierce's PR-specific authorisation. No merge or repository change was performed.