Skip to content

fix(evolution): do not reuse stale CBOR chunk sizes on re-encode (#575) - #634

Open
Kshot3000 wants to merge 1 commit into
IntersectMBO:mainfrom
Kshot3000:fix/cbor-stale-chunk-sizes
Open

Kshot3000 wants to merge 1 commit into
IntersectMBO:mainfrom
Kshot3000:fix/cbor-stale-chunk-sizes

Conversation

@Kshot3000

Copy link
Copy Markdown

Fixes #575

Problem

When a byte or text string was decoded as indefinite-length chunks, the captured format kept the chunk sizes. If the value then changed and was encoded with that format, the encoder wrote the old chunk sizes over the new value — valid CBOR carrying different content, with no error:

const { format } = CBOR.fromCBORHexWithFormat("5f4201024103ff") // h'010203' as chunks 2 + 1

CBOR.toCBORHexWithFormat(new Uint8Array([0xaa]), format)
// before: 5f42aa004100ff — decodes as h'aa0000' (zero padded)
// after: decodes back to h'aa'

CBOR.toCBORHexWithFormat(new Uint8Array([1, 2, 3, 4, 5]), format)
// before: 5f4201024103ff — decodes as h'010203' (tail dropped)
// after: decodes back to h'0102030405'

Text strings behaved the same way: "x" under the format of "hij" decoded as "x\0\0", and a change could split a UTF-8 character. This is the metadata-hash mismatch from the issue: the SDK hashed the intended value but wrote different bytes.

Fix

In CBOR.ts, a captured indefinite chunk layout is now reused only when it still fits the value, per the issue's prescribed fix:

  • bytes: the chunk sizes add up to the value's length
  • text: the same, and every chunk ends on a UTF-8 character boundary (the next byte is not a continuation byte)

Otherwise the encoder falls back to the default encoding for that string, as integer widths already do under the preservation spec. The definite branch is unchanged — encodeIntHeader already falls back when a preserved width no longer fits. An unchanged value, and a same-length change whose chunks still align, keep their captured layout byte for byte.

Tests

Red first, one filter at a time — the shorter-bytes test failed with expected Uint8Array[170, 0, 0] to deeply equal Uint8Array[170], the longer-bytes test with the tail dropped, and the shorter-text test with 'x\0\0'.

9 new tests in CBOR-with-format.test.ts:

  • bytes: shorter and longer values under a chunked format decode back to themselves; an unchanged value stays byte-identical; a same-length changed value still reuses its chunk layout
  • text: shorter and longer strings decode back to themselves; "héllo" under a mismatched format decodes back to itself; a same-length "xé" whose 2-byte first chunk would split the 2-byte é falls back; an unchanged string stays byte-identical

File: 33/33 pass. Full suite: 1687 passed / 75 skipped / 0 failed, test files 101 passed / 1 skipped of 102 (main baseline 1678 + 9 new). tsc --noEmit and eslint clean. Prettier: CBOR.ts fails the check on unmodified main too, and my version's prettier diff-line count exactly equals main's (304 = 304), so the change adds no new formatting drift; the test file is prettier-clean.


Independent builder — not affiliated with the Cardano Foundation, IOG or Intersect.

@Kshot9000 (https://x.com/kshot9000) · ADA: addr1q8hnl6vl5a6k3rw3n5g3jtte696zcl76kfatzv7gpswa9r0dj7fma6klq55y4ffm7tf0em09udnyhuk4ah92pl5x9jpqjae44v

cc @cardano-foundation @IntersectMBO

…ersectMBO#575)

A captured indefinite chunk layout was replayed over a changed byte or
text string regardless of the new value's length: a shorter value was
zero-padded to the old chunk total, a longer value lost its tail, and a
text change could split a UTF-8 character — valid CBOR carrying
different content, with no error.

The layout is now reused only when it still fits: for bytes the chunk
sizes must add up to the value's length; for text the same, and every
chunk must end on a UTF-8 character boundary. Otherwise the encoder
falls back to the default encoding for that string, as integer widths
already do. Unchanged values and same-length aligned changes keep
their captured layout byte for byte.

Tests: 9 new checks in CBOR-with-format.test.ts (shorter/longer bytes
and text, UTF-8 split at equal length, byte-identical controls).
Full suite: 1687 passed / 75 skipped / 0 failed.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CBOR: preservation-aware encode reuses stale chunk sizes, writing different content

1 participant