Skip to content

spec(0.9.0): a producer MUST escape U+0000–U+001F — the requirement was inherited by implication only - #12

Merged
coderoast-dev merged 1 commit into
mainfrom
spec/encoding-conformance-clause
Aug 28, 2026
Merged

coderoast-dev merged 1 commit into
mainfrom
spec/encoding-conformance-clause

Conversation

@coderoast-dev

Copy link
Copy Markdown
Collaborator

What changes

SPEC.md §2 gains an explicit Encoding clause: a producer MUST escape any character in the range U+0000–U+001F occurring in a string — member names and values alike, at every depth, extensions included — using RFC 8259's two-character escape where one is defined (\b, \f, \n, \r, \t), otherwise \uXXXX. It MUST NOT emit the raw character.

SPEC.md §8 gains a fifth conformance clause, so the requirement is something a conformance suite can cite rather than infer, and §8's closing paragraph prices clause 5 the way it already prices clause 4. §16.4 gains a pointer at the same clause: its existing rule constrains cube coord values by type only (a string or an array of strings) and reads as though type were the whole constraint — and axis values come from the observed stream, which is exactly where a control character arrives.

CHANGELOG.md gains a ### Clarified entry under the unreleased 0.9.0.

No schema file changes.

Why this is Editorial, and carries no version bump

Editorial under GOVERNANCE.md §2. The reason is not that the change is small — it is that no valid document becomes invalid.

§1 already defines a MetaLog as "A JSON document", and RFC 8259 §7's unescaped production excludes U+0000–U+001F. So a byte sequence carrying a raw control character inside a string was never a JSON text, and therefore was never a MetaLog — before this clause and after it, identically. No conformant document's bytes change; no producer that was conformant becomes non-conformant.

The conformance bar did not move. What moved is the instrument's reach. The requirement was inherited by implication only: the spec named neither RFC 8259 nor escaping anywhere in its text, so an implementer had no clause to point at when rejecting such a document, and no clause telling them their producer must escape.

That is why this rides the already-unreleased 0.9.0 rather than waiting for the 0.10.0 RFC window. An RFC window exists to protect implementers from a moving bar; this bar did not move.

What an implementer must change

If your producer is correct: nothing.

  • Consumers: nothing. Your parser already rejected these documents — it had to, they are not JSON. Clause 5 does not ask you to add a check; it tells you what to call the rejection you already perform, so you can attribute it.
  • Producers: audit your write sites if you hand-build JSON by concatenation, or your serializer treats control-character escaping as opt-in rather than the default. Both are common. The second is the dangerous one, because taking the default is silent — nothing in your code reads as a decision.

This was our own defect, and it is why the requirement earns a clause instead of being left to RFC 8259. An audit of the reference implementation's JSON write sites — precisely the audit this section asks for — found five of seven egress writers taking a serializer default that does not escape. Two sites set the flag; five did not. That was not seven independent judgements: it was one judgement made twice, and five defaults taken. The remedy had to be structural rather than per-site — one escape-forcing egress wrapper per package, with the unsafe write forms made unreachable rather than merely discouraged, and a source gate that keeps them that way.

Why the rule is MUST ESCAPE, never MUST NOT EMIT

The clause constrains encoding; it takes no position on content. A log-derived value that legitimately carries U+0000 is emitted as \u0000 and round-trips losslessly. Forbidding the content would put this standard in the business of adjudicating the values its producers observe, which it has no basis to do.

Why it earns a clause of its own

The failure is silent in a specific way. A violating document is not JSON, so a consumer's parser rejects it before any schema runs — and that rejection is indistinguishable from a truncated or corrupt file. The two have opposite remedies: fix the producer's escaping, or re-fetch the file. So §8's closing paragraph now requires a validator that decides clause 5 to report it as a producer failure, held apart from an unreadable corpus.

Clause 5 is not reachable from the schema — and for a stronger reason than clause 4. Clause 4 is unreachable because maxItems takes a constant while the bound is a sibling field's value; clause 5 is unreachable because no parser ever hands a non-JSON byte sequence to a schema validator, so no keyword can fire at all.

It is decidable from the document's bytes, with no parse: RFC 8259 §2 admits only U+0009, U+000A, U+000D and U+0020 as inter-token whitespace, so 29 of the 32 characters in U+0000–U+001F may not appear raw anywhere in a JSON text, and only those three remaining need string context to judge.

Verification

Measured on this branch:

  • conformance/metalog_validate.py --selftest → 22/22 fixtures, 8 controls armed, exit 0
  • gating run over schema/metalog.v0.example.json → exit 0, VERDICT: CONFORMANT

Both are unchanged from the pre-edit baseline, which is what a declaratory clause requires: a change that moved either number would not have been editorial.

Base: main at b4724fb.

…d the implication is not citable

SPEC.md named neither RFC 8259 nor escaping anywhere in its text. §2 said "a
MetaLog MUST be a single JSON object" and §16.4 constrained cube coordinate
values by TYPE only ("a string or an array of strings"), so an external
implementer had no clause to cite when rejecting a document carrying a raw
U+0000-U+001F inside a string, and no clause telling them their producer must
escape. Verified at HEAD b4724fb by grep over SPEC.md for
`RFC 8259|ECMA-404|control char|control byte|escape|escaped|unescaped|U+00`:
zero hits.

§2 gains an Encoding clause and §8 a fifth conformance clause. The rule is MUST
ESCAPE, never MUST NOT EMIT: this standard constrains ENCODING and takes no
position on CONTENT. A log-derived value legitimately carrying U+0000 is emitted
escaped and round-trips losslessly; forbidding the content would put the format
in the business of adjudicating the values its producers observe.

EDITORIAL under GOVERNANCE.md §2, and no version bump. §1 already defines a
MetaLog as "a JSON document", and RFC 8259 §7 excludes U+0000-U+001F from
`unescaped`, so a byte sequence carrying one raw inside a string was never a
JSON text and therefore never a MetaLog. No conformant document's bytes change
and no valid document becomes invalid. The conformance bar did not move; what
moves is that the requirement became citable.

§8's closing paragraph prices clause 5 the way it already prices clause 4: not
reachable from the schema, and for a stronger reason -- a violating document is
not JSON, so no parser hands it to a schema validator and no keyword can fire.
It is decidable from the BYTES with no parse: RFC 8259 §2 admits only U+0009,
U+000A, U+000D and U+0020 between tokens, so 29 of the 32 characters in the
range may not appear raw anywhere in a JSON text and only three need string
context. A validator deciding it MUST report a PRODUCER failure and hold it
apart from an unreadable corpus -- the two present with the same parser error
and have opposite remedies.

Measured after the edit: `metalog_validate.py --selftest` 22/22 fixtures, 8
controls armed, exit 0; the gating run over `schema/metalog.v0.example.json`
exits 0, VERDICT CONFORMANT. Both unchanged from the pre-edit baseline, as a
declaratory clause requires.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@coderoast-dev
coderoast-dev merged commit 0a4fbf6 into main Aug 28, 2026
1 check passed
@coderoast-dev
coderoast-dev deleted the spec/encoding-conformance-clause branch September 19, 2026 08:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant