spec(0.9.0): a producer MUST escape U+0000–U+001F — the requirement was inherited by implication only - #12
Merged
Conversation
…d the implication is not citable
SPEC.md named neither RFC 8259 nor escaping anywhere in its text. §2 said "a
MetaLog MUST be a single JSON object" and §16.4 constrained cube coordinate
values by TYPE only ("a string or an array of strings"), so an external
implementer had no clause to cite when rejecting a document carrying a raw
U+0000-U+001F inside a string, and no clause telling them their producer must
escape. Verified at HEAD b4724fb by grep over SPEC.md for
`RFC 8259|ECMA-404|control char|control byte|escape|escaped|unescaped|U+00`:
zero hits.
§2 gains an Encoding clause and §8 a fifth conformance clause. The rule is MUST
ESCAPE, never MUST NOT EMIT: this standard constrains ENCODING and takes no
position on CONTENT. A log-derived value legitimately carrying U+0000 is emitted
escaped and round-trips losslessly; forbidding the content would put the format
in the business of adjudicating the values its producers observe.
EDITORIAL under GOVERNANCE.md §2, and no version bump. §1 already defines a
MetaLog as "a JSON document", and RFC 8259 §7 excludes U+0000-U+001F from
`unescaped`, so a byte sequence carrying one raw inside a string was never a
JSON text and therefore never a MetaLog. No conformant document's bytes change
and no valid document becomes invalid. The conformance bar did not move; what
moves is that the requirement became citable.
§8's closing paragraph prices clause 5 the way it already prices clause 4: not
reachable from the schema, and for a stronger reason -- a violating document is
not JSON, so no parser hands it to a schema validator and no keyword can fire.
It is decidable from the BYTES with no parse: RFC 8259 §2 admits only U+0009,
U+000A, U+000D and U+0020 between tokens, so 29 of the 32 characters in the
range may not appear raw anywhere in a JSON text and only three need string
context. A validator deciding it MUST report a PRODUCER failure and hold it
apart from an unreadable corpus -- the two present with the same parser error
and have opposite remedies.
Measured after the edit: `metalog_validate.py --selftest` 22/22 fixtures, 8
controls armed, exit 0; the gating run over `schema/metalog.v0.example.json`
exits 0, VERDICT CONFORMANT. Both unchanged from the pre-edit baseline, as a
declaratory clause requires.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes
SPEC.md§2 gains an explicitEncodingclause: a producer MUST escape any character in the range U+0000–U+001F occurring in a string — member names and values alike, at every depth,extensionsincluded — using RFC 8259's two-character escape where one is defined (\b,\f,\n,\r,\t), otherwise\uXXXX. It MUST NOT emit the raw character.SPEC.md§8 gains a fifth conformance clause, so the requirement is something a conformance suite can cite rather than infer, and §8's closing paragraph prices clause 5 the way it already prices clause 4. §16.4 gains a pointer at the same clause: its existing rule constrains cubecoordvalues by type only (a string or an array of strings) and reads as though type were the whole constraint — and axis values come from the observed stream, which is exactly where a control character arrives.CHANGELOG.mdgains a### Clarifiedentry under the unreleased0.9.0.No schema file changes.
Why this is Editorial, and carries no version bump
Editorial under
GOVERNANCE.md§2. The reason is not that the change is small — it is that no valid document becomes invalid.§1 already defines a MetaLog as "A JSON document", and RFC 8259 §7's
unescapedproduction excludes U+0000–U+001F. So a byte sequence carrying a raw control character inside a string was never a JSON text, and therefore was never a MetaLog — before this clause and after it, identically. No conformant document's bytes change; no producer that was conformant becomes non-conformant.The conformance bar did not move. What moved is the instrument's reach. The requirement was inherited by implication only: the spec named neither RFC 8259 nor escaping anywhere in its text, so an implementer had no clause to point at when rejecting such a document, and no clause telling them their producer must escape.
That is why this rides the already-unreleased
0.9.0rather than waiting for the0.10.0RFC window. An RFC window exists to protect implementers from a moving bar; this bar did not move.What an implementer must change
If your producer is correct: nothing.
This was our own defect, and it is why the requirement earns a clause instead of being left to RFC 8259. An audit of the reference implementation's JSON write sites — precisely the audit this section asks for — found five of seven egress writers taking a serializer default that does not escape. Two sites set the flag; five did not. That was not seven independent judgements: it was one judgement made twice, and five defaults taken. The remedy had to be structural rather than per-site — one escape-forcing egress wrapper per package, with the unsafe write forms made unreachable rather than merely discouraged, and a source gate that keeps them that way.
Why the rule is MUST ESCAPE, never MUST NOT EMIT
The clause constrains encoding; it takes no position on content. A log-derived value that legitimately carries U+0000 is emitted as
\u0000and round-trips losslessly. Forbidding the content would put this standard in the business of adjudicating the values its producers observe, which it has no basis to do.Why it earns a clause of its own
The failure is silent in a specific way. A violating document is not JSON, so a consumer's parser rejects it before any schema runs — and that rejection is indistinguishable from a truncated or corrupt file. The two have opposite remedies: fix the producer's escaping, or re-fetch the file. So §8's closing paragraph now requires a validator that decides clause 5 to report it as a producer failure, held apart from an unreadable corpus.
Clause 5 is not reachable from the schema — and for a stronger reason than clause 4. Clause 4 is unreachable because
maxItemstakes a constant while the bound is a sibling field's value; clause 5 is unreachable because no parser ever hands a non-JSON byte sequence to a schema validator, so no keyword can fire at all.It is decidable from the document's bytes, with no parse: RFC 8259 §2 admits only U+0009, U+000A, U+000D and U+0020 as inter-token whitespace, so 29 of the 32 characters in U+0000–U+001F may not appear raw anywhere in a JSON text, and only those three remaining need string context to judge.
Verification
Measured on this branch:
conformance/metalog_validate.py --selftest→ 22/22 fixtures, 8 controls armed, exit 0schema/metalog.v0.example.json→ exit 0,VERDICT: CONFORMANTBoth are unchanged from the pre-edit baseline, which is what a declaratory clause requires: a change that moved either number would not have been editorial.
Base:
mainatb4724fb.