Skip to content

spec(0.9.0): the size budget is a formula, and §8 clause 4 stops being unreachable - #5

Merged
coderoast-dev merged 1 commit into
mainfrom
spec/size-budget-is-a-formula
Aug 23, 2026
Merged

coderoast-dev merged 1 commit into
mainfrom
spec/size-budget-is-a-formula

Conversation

@coderoast-dev

Copy link
Copy Markdown
Collaborator

Additive under GOVERNANCE.md §2 — new optional fields only. No field changes type, becomes required, or is removed; no conformant 0.8.0 document becomes invalid.

What was wrong

§11 priced one of five variable-length blocks and indexed the whole envelope on k. Measured on this repo's own example document (compact JSON):

Block Bytes/entry Bounded by Was in §11?
stats.top_k 99–177 (id-only) top_k_size the only one
stats.reservoir 165–296 reservoir size no
behavior.top_ngrams ~121 top_ngrams_size no
behavior.branching ~107 nothing no
cube.cells 27–99 §16.10 budget (4096) no

A reservoir entry costs 1.5–2.5× a top_k entry, and the cube's declared budget alone permits ~252 KB — an order of magnitude above the table's largest row. The table was not wrong on its own block; it was silent about the four that dominate.

What this changes

§11 becomes a formula (informative section, no normative effect): fixed + Σ (declared cap × measured per-entry cost), with the per-entry costs measured on schema/metalog.v0.example.json and three worked configurations. A consumer prices its own documents instead of trusting a number that rots the first time a producer moves a knob.

Three caps gain a field to be declared in — stats.reservoir_size, behavior.branching_size, cube.cell_budget. This is what made the formula unwritable before. retention_profile (§2.4) does not cover it: it is opaque, and it answers comparability, never size.

behavior.branching keeps its uncapped posture, now stated. It is the one block the spec places no cap on; omitting the field means the producer declares no cap, which is legal, and a size-constrained consumer must read the omission that way rather than assume a default.

§11.5 scopes the "≤ 4 KB / 1M lines" target to the stats-only document. The target is unchanged and still reached at top_k_size ≤ 32 inline / ≤ 64 id-only. What changes is that its scope is stated instead of implied.

§8 clause 4 generalises from top_k to every declared cap, and stops being decorative. §8 says "the schema is the test" — and maxItems appears zero times in either schema. That is not a schema lag but a structural impossibility: maxItems takes a constant, while the bound here is the value of a sibling field. So the clause is now checked where it can be, in the shipped validator, which until this PR printed its own blindness on every run.

The check

conformance/metalog_validate.py gains CAP-EXCEEDED (exit 1). The cap/array pair set is derived from the schema — an integer <x>_size whose sibling <x> is an array — so a pair added tomorrow is checked on arrival; a hand-kept list would rot silently. cube.cell_budget is the single declared exception (named for §16.10's BUDGET, not for its array). Objects under extensions are skipped: vendor space is not this standard's to adjudicate.

Mutation-proven. Disabling the check reds exactly 1 of 13 fixtures with actual-vs-expected, the other 12 staying green — nothing else was holding this property. The new fixture is schema-valid and cap-violating, which is precisely the case a schema-only validator reports CONFORMANT; it violates at two locations (stats and behavior) so a top_k-only checker reds, and carries a vendor shard_size/shard pair under extensions that must not be reported.

For the editor

  • The v0.9.0 line in SPEC.md's header, the example's metalog_version, and the CHANGELOG's — unreleased date are yours to confirm. 0.9.0's other planned members (run_outcome, reservoir_delta, diff-root extensions per adr/0003) append to the same CHANGELOG section.
  • RATIONALE.md and adr/0002 still quote the old ~150 B/entry / ~10 KB figures. Left untouched on purpose: both are records of a decision at the time it was made, and ~150 B sits inside the measured inline range. Say the word if you want them repointed at §11.2.
  • One live cross-reference was repointed: §3.6.1's "doubling top_k grows the envelope by ~10 KB inline" is now "~9 KB id-only / 13–14 KB inline, at §11.2's measured costs".
  • README.md's value-prop paragraph already refused to publish a compression ratio ("that is a target and a bound, not a measured ratio"). This PR keeps that and adds the scope the paragraph was missing.

Gates: --selftest 13/13 with declared-cap-violation armed · schema/metalog.v0.example.json CONFORMANT · both schemas valid Draft 2020-12.

…nreachable

§11 priced ONE of five variable-length blocks and indexed the whole envelope on
`k`. Measured on this repo's own example document, compact JSON: a `reservoir`
entry costs 165-296 B against a `top_k` entry's 99-177 (id-only), and §16.10's
closed-cell budget of 4096 permits ~252 KB — an order of magnitude above the
table's largest row. The table was not wrong on its own block; it was silent
about the four that dominate.

The replacement is a sum whose every term is a DECLARED cap times a measured
per-entry cost, so a consumer prices its own documents instead of trusting a
number that rots the first time a producer moves a knob. Three of the five caps
had no field to be declared in, which is why the formula could not be written
before: `stats.reservoir_size`, `behavior.branching_size` and `cube.cell_budget`
are added (optional, additive). `retention_profile` does not cover this — it is
opaque, and it answers comparability, never size.

`behavior.branching` keeps its uncapped posture, now stated: omitting the field
means the producer declares NO cap. Legal, and a consumer must read it that way
rather than assume a default.

§11.5 scopes the "4 KB / 1M lines" target to the `stats`-only document. The
target is unchanged; what changes is that its scope is stated instead of implied.

And clause 4 stops being decorative. §8 says "the schema is the test", and
`maxItems` appears ZERO times in either schema — not a lag, a structural
impossibility: `maxItems` takes a constant and the bound is the value of a
sibling field. So the clause is checked where it CAN be, in the shipped
validator, which until now printed its own blindness on every run. The cap/array
pair set is derived from the schema (`<x>_size` whose sibling `<x>` is an array),
so a pair added tomorrow is checked on arrival; `cube.cell_budget` is the single
declared exception. `extensions` is skipped — vendor space is not ours to judge.

Mutation-proven: disable the check and exactly 1 of 13 fixtures reds with
actual-vs-expected, the other 12 green — nothing else was holding this property.
The fixture is SCHEMA-VALID and cap-violating, which is the case a schema-only
validator reports CONFORMANT.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@coderoast-dev
coderoast-dev merged commit f443495 into main Aug 23, 2026
1 check passed
@coderoast-dev
coderoast-dev deleted the spec/size-budget-is-a-formula branch September 19, 2026 08:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant