feat: keep file bytes in object storage

[quality-runtime]

Adds the files half of evidence: a client uploads bytes straight to an S3-compatible bucket, and PostgreSQL decides whether they ever become an attachment. This is the storage work #8 split out and promised, so ADR 0020's links to ADR 0013 resolve again.

Evidence attachments are a large part of what the CRA expects a manufacturer to keep for ten years (Art. 13(13)), and a mounted volume offers nothing for an obligation that long — no versioning, no object lock, no lifecycle policy, no replication, nothing to point an auditor at. A bucket offers all of them as configuration an operator already knows how to buy.

The routes

Method	Path	What it does
POST	/evidence/{id}/file-uploads	Prepares an upload. Answers 201 with a presigned PUT URL, good for fifteen minutes
PUT	/file-uploads/{id}/completion	Attaches what arrived. Idempotent on the upload id
GET	/files/{id}	303 to a signed GET, good for a minute

Evidence responses gain a files field. A file is at most 25 MiB and evidence carries at most twenty.

The lifecycle

sequenceDiagram
    participant C as Client
    participant R as Runtime
    participant S as Bucket
    participant P as PostgreSQL
    C->>R: POST …/file-uploads
    R->>P: insert file_upload (intent, 15 min)
    R-->>C: 201 + presigned PUT
    C->>S: PUT uploads/upl_…
    C->>R: PUT …/completion
    R->>S: HEAD uploads/upl_… (cheap filter)
    R->>S: COPY → files/fil_…, if-match the inspected tag
    R->>S: GET files/fil_…, if-match the copy's own tag
    Note over R: size + SHA-256, one pass
    R->>P: insert file, claim the intent
    R->>S: DELETE uploads/upl_…
    R-->>C: 200 file

Loading

PostgreSQL owns finality, authorization, idempotency and audit; the bucket owns bytes. No database connection is held while the store is being talked to — concurrency.test.ts runs eight completions through a pool of two to prove it.

Both durable facts come from the permanent object

A presigned PUT constrains the method and the key, not the headers. A client can store bytes as Content-Encoding: gzip; the store keeps that as metadata and fetch hands the response back decompressed, while the copy moves the stored bytes and leaves the encoding behind. Hashing the staged object would therefore record a checksum of bytes nobody keeps, and the first verify:files would call a new file altered. The entity tag is no help — it validates content, not metadata.

So the row is built from reading files/{fileId} back, pinned to the tag the copy itself reported, and the size and SHA-256 come from that one stream.

Uncertainty keeps bytes

The two mistakes are not equal: bytes nothing names cost a sweep, and removing bytes a committed row names cannot be undone. A promoted object is removed only where the rollback is certain from the handler's own control flow — every domain refusal is returned as a value, and the one exception is the handler's own. An unexpected driver error keeps the bytes, because such an error does not say whether the commit was made durable before it arrived.

Note

An earlier draft classified SQLSTATEs into "the server refused this statement" and "this says nothing". It was deleted rather than corrected: pg reports PostgreSQL's localized severity and not the non-localized field, and a list of codes that is not exhaustive fails in the direction that destroys evidence. It also only ever decided the unanticipated failures — exactly the ones this asymmetry says to keep.

Details worth a reviewer's eye

• An intent is not evidence and reserves nothing. file_upload is infrastructure state, in no history, cascading from its evidence. Evidence attested between prepare and completion refuses the completion.
• The window is the database's to keep. The runtime holds UPDATE on file_id alone, so which evidence an upload is for and when it expires cannot move while the store is being read. The completion policy judges by clock_timestamp() — now() is frozen at transaction start and would admit an upload that expired while waiting for the evidence lock.
• Each attempt promotes to a fresh file id, so two completions of one upload cannot overwrite each other, and the loser returns the winner's file.
• The session cookie carries Path=/api and no Domain, and startup refuses a bucket whose URL is this host inside that path — judged on {endpoint}/{bucket} together, since a bucket named api lands a download there as surely as an endpoint path does. Depth, not a boundary: production gives the store a hostname of its own.
• No permanent key is ever signed for writing. A signed URL outlives the request, and one for files/{fileId} would be a standing licence to replace an attested record's bytes.

Operating it

bun run verify:files recomputes every checksum against the bucket. bun run reclaim:storage compares the bucket against the rows and reports what nothing claims — removing only with --remove, leaving anything written in the last day, ignoring keys it did not issue, and refusing outright when the database holds no files at all.

Storage compatibility is a register rather than a claim: CI asks MinIO on every push, and .github/workflows/storage-compatibility.yml asks Cloudflare R2 on a schedule. AWS and B2 are configurations expected to work, not demonstrated ones.

Decisions

ADR 0021 records the design. It supersedes ADR 0013's upload shape and mounted-volume adapter — 0013's asymmetry stands, its reading of the failure boundary does not — and builds on ADR 0016 (what verify:files is for) and ADR 0012 (attachments are final once attested). ADR 0020 gains the races this adds.

Verification

On the committed tree, clean:

bun run check                                     # format, lint, types
bun run test                                      # 685 passing, 11 skipped
uvx --from 'reuse[charset-normalizer]' reuse lint
TEST_DATABASE_URL=…_test bun run test apps/server/concurrency.test.ts   # 31, real PostgreSQL 18.6

Each new guarantee was checked by removing it and watching a test fail: the clock_timestamp() policy, the claim's row count, the permanent-object measurement, the storage isolation check, the list prefix guard, the cookie attributes, and the audit payload's five fields.

Note

MinIO removed minio/minio and minio/mc from Docker Hub, so CI could not start a store at all — repository does not exist. Both now come from quay.io, pinned: that line has stopped moving, because MinIO's ongoing product is AIStor, which needs a license file and is not open-source. docs/development.md explains the choice so nobody "upgrades" it back.

Then the MinIO run earned its keep and caught a real provider bug, which is exactly the composition the integration suite exists to check. Promotion read the entity tag out of CopyObjectResult and handed it straight back as If-Match; AWS escapes the quotes as " and Go's encoding/xml — so MinIO — writes ", and only the first was decoded. Every permanent read answered 412 and every completion failed with it. Both are decoded now, and the tag is re-quoted rather than passed through, since a bare token is not an entity tag (RFC 9110). Five spellings are a table in objects.test.ts, and the integration suite compares the copy's tag with the one a HEAD reports before using it as a precondition — so a future provider disagreement names the two strings instead of answering 412.

Not here

• Create-never-replace promotion. The copy is conditional on the source and unconditional on the destination; sixteen base-36 characters is the only thing preventing a collision. The right contract is written down in ADR 0021 for when promote is next revisited.
• Positive database↔bucket ownership for destructive reclamation — ownership is decided by absence, which the empty-database refusal only partly covers.
• Upload quotas. A member can prepare as many uploads as they like; a lifecycle rule bounds how long abandoned bytes stay, not how many arrive. That is a quota, and it belongs where membership is decided.
• A maintenance command for pruning expired intents without touching objects, and checksum → sha256.