fix(cpp): bound TS_2DIFF block headers by the page value count - #931
Open
kkzi wants to merge 1 commit into
Open
Conversation
A zero-bit-width block consumes no packed bytes, so byte availability cannot bound its writeIndex. With no page-value-count bound in scope, a 16-byte crafted block header (writeIndex = INT32_MAX, bitWidth = 0) made the decoder emit 2^31 values and hang the reader loops. Plain int32/int64 TS_2DIFF pages (no page metadata at all) and Form 1 float/double pages (metadata carries no value count) were both affected; Form 2/3 pages were already bounded. Root-cause fix where all readers route through: Decoder gains a set_page_value_count() hook (default no-op), TS2DIFFDecoder bounds writeIndex by it, and every page-decode site plumbs the page's point count into the decoder: ChunkReader::decode_cur_page_data (time and value), AlignedChunkReader::decode_cur_time_page_data / decode_cur_value_page_data, and AlignedChunkReader:: decode_time_page_with (sequential and worker-pool time decode, whose count is captured from the page statistic at plan time — row_begin/ row_end cannot serve, being the filtered sub-span on boundary pages). The aligned value-page predecode path is already bounded by the page's non-null count. Float/Double decoders apply the plumbed count in ensure_page_meta for Form 1 pages; a page declaring zero points rejects every block rather than disabling the bound. Known residuals, both rooted in the format rather than this change: - Single-page chunks carry no page statistic on the wire (the page header is deserialized without statistics when the chunk has one page), so those pages keep decoding unbounded. - The bound is only as tight as the page statistic itself, which a crafted file also controls; it enforces the block-vs-page invariant, not an absolute work limit. The single-page recovery loops in RestorableTsFileIoWriter likewise keep relying on read errors. Java grounding: DeltaBinaryDecoder allocates new int[packNum], so a writeIndex = INT32_MAX page is not valid Java input either.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A crafted 16-byte TS_2DIFF block header (
writeIndex = INT32_MAX,bitWidth = 0) makes the decoder emit up to 2^31 values, hanging the reader loops (while (decoder->has_remaining(...))). Zero-width blocks consume no packed bytes, so byte availability cannot boundwriteIndex. Affected: plain int32/int64 pages (no page metadata at all) and Form 1 float/double pages (metadata carries no value count). Form 2/3 pages were already bounded.Fix
Bound block headers by the page's point count from the page header statistic:
Decodergains aset_page_value_count()hook (default no-op).TS2DIFFDecoderrejects blocks withwriteIndex + 1 > page count(E_DECODE_ERR).ChunkReader::decode_cur_page_data(time + value),AlignedChunkReader::decode_cur_time_page_data/decode_cur_value_page_data/decode_time_page_with(sequential and worker-pool). The aligned value-page predecode path is already bounded by its non-null count.Java grounding:
DeltaBinaryDecoderallocatesnew int[packNum], so such pages are not valid Java input either.Tests
Known residuals (format-level, documented in the commit)