Resolve a search page's argMax for its own keys, not every row past the cursor

[zaoxing]

What

Every search() page cost about as much as the whole catalog, whatever its size. A 38-row page over a 198k-row catalog read all 198,207 rows: 59 MiB read, 263 MiB of memory, ~150 ms.

Cause. Search ran a single GROUP BY … ORDER BY … LIMIT over *_capture_raw. That meant ClickHouse built the full 27-column argMax resolution tuple for every group past the cursor, and only then did LIMIT keep limit + 1 of them. The keyset comparison below is not usable by the primary-key index:

(tenant_id, experiment_id, run_id, captured_at_ns, capture_id) > (…)

EXPLAIN indexes = 1 reports 26/26 granules, so nothing was pruned.

Attributing the time in one page:

Variant	Time
filter only (count())	22 ms
filter + GROUP BY of the 5 key columns	37 ms
the real page query	157 ms

The argMax tuple accounts for about three quarters of it.

The change

The native reader now chooses a page's keys first, then resolves the aggregate for those keys only:

SELECT … argMax(tuple(…), …) FROM raw
WHERE <filters> AND (keys) IN (
    SELECT keys FROM raw WHERE <filters> GROUP BY keys ORDER BY keys LIMIT n)
GROUP BY keys ORDER BY keys LIMIT n

Both queries carry every filter, so the groups and each group's resolution are unchanged. This relies on the design doc's existing rule that every descriptor field except the locator is immutable per capture, which already justifies the pre-aggregation filters. The design doc now describes the shape.

Native only. The Python storage capture path stays in the tree only as an unused reference, so this PR doesn't touch the Python reader. That turns out to strengthen the test: the Python reference reader keeps the single-phase shape, so the parity suite now checks the new native query against the old one.

Evidence

Paging through one 196,608-capture run on a 204k-capture catalog with the native reader, via the conformance_catalog driver built from main and from this branch:

Page size	Before	After	Speedup
10,000	5.53 s	3.31 s	1.7×
2,000	20.80 s	8.02 s	2.6×
500	75.57 s	24.11 s	3.1×

Identical ids in identical order at every page size, by checksum over the full id sequence. The checksum also matches the Python reader's walk of the same run, so both implementations agree on all 196,608 ids.

The reader suites pass 155/155, up from 154 on main plus the new guard: cpu, live reader, snapshot, end-to-end, and native/Python reader parity. The conformance drivers were rebuilt first, so no parity case skipped.

The guard, test_native_pages_resolve_argmax_for_their_own_keys_and_match_python, lives in the parity suite and fails on main. It checks two things:

• Result. It walks the native reader in pages of 3 across a superseded version (four captures re-described from a different pack at a later index_version, where argMax decides the answer) and compares every row with the single-phase Python reader.
• Shape. It reads the queries ClickHouse actually received from the native driver out of system.query_log, and requires the key-tuple subquery on each.

What it doesn't do

Each page still scans the rows past the cursor. The keyset tuple stays invisible to the index, which leaves a floor of about 61 ms per page here. I measured that a redundant bound such as experiment_id = … or captured_at_ns >= … does prune (14/26 granules), but it didn't reduce the time, because aggregation dominated. Removing the floor needs index-usable cursor bounds, which is a separate change to cursor encoding.

After independent review (5106083)

The reviewer walked a 1.1M-row catalog with unpublished packs, supersession, a mirror store, pins and filters. Old and new queries gave identical results in every walk. The new query was 2.5–4× faster on unfiltered and cursor walks (for example 127.8 s → 31.9 s).

• A test that catches the two queries filtering differently. In test_a_page_walk_skips_unpublished_keys_without_ending_early, unpublished keys sort between member keys across page boundaries. A full walk of 2-row pages must return exactly the member captures, with and without a hook filter. Two mutations each fail it: dropping snapshot membership from the inner query, and dropping the hook filter from it. The previous tests passed both mutations, and the walk silently ended early.
• The row budget can double. max_rows_to_read covers the whole statement, and both queries read capture_raw, so a page can read up to twice the rows it did before. This is documented in docs/capture-storage-design.md and in reader.cpp. Size max_rows_to_read for two passes.
• Stale comment fixed: it no longer claims the native query has the same shape as the Python reader.
• Not changed: the filter text now appears in both queries, doubling the statement size. A hook list near the 128-name limit can exceed ClickHouse's default 256 KiB max_query_size.

Brought up to date with main (#149–#152): pytest -m cpu 2520 passed. The reader parity, capture storage, snapshot and chain live suites gave 100 passed, plus the lease suite (54) before the last merge.

For #156: it moves snapshot membership into FROM snapshot(). When it is rebased onto this, the inner key query must also read from snapshot(), and add WHERE clauses only when there are filters. Keeping FROM capture_raw WHERE clauses would silently drop the snapshot, or fail with no filters. The new test catches both.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.