probe: does vectorizing r2sleigh's OpColumns scans pay? Measured — mostly no - #308
Conversation
…stly no `r2il::columns` was laid out for this mask surface and named `eq_u32_to_mask` / `masked_strided_group_sum` in its own module docs, while deliberately taking no ndarray dependency and declining to claim the SIMD was worth it: "a profiling question nobody has answered." This answers it from the consumer side, on a real x86-64 lift (12 408 p-code ops from the win32-census fixture) rather than a synthetic stream. Four arms compute an identical mask — scalar over the native u8/u64 columns, 4 predicates + 3 mask_and_assign, the same with one mask_ternlog::<AND3>, and the _under chain — with a bit-for-bit equivalence gate before any timing, swept from 256 ops to 3.2 M. Findings, all in the module doc with the table: - The crossover is low: the mask arms win to ~50 K ops and LOSE from ~200 K (0.75-0.80x at 3.2 M). The scalar arm reads 9 B/op; the mask arms read 12 B/op of widened columns plus four mask buffers. - The ternlog fusion is not the lever here — TERN and AND are within noise at every span. The cost is the four passes over value columns, not the three combines fusion removes. Fusion pays when a caller already HOLDS the masks. - `_under` is the arm that survives scale: no separate combine, no extra buffers, best at the real size (1.42x) and nearest parity at 3.2 M. - The ratio is favourable exactly where the absolute time is irrelevant: 8.9 us scalar vs 6.4 us vectorized for a whole-census scan, on a binary whose SLEIGH lift costs milliseconds. Two primitive gaps the probe had to work around, and both are findings: no u8 comparator (the columns are Vec<u8>; the narrowest value type is u32, so a consumer pays a 4x widened copy) and no u64 RANGE comparator (only exact/ternary match; the ordered family stops at i32). Measured, 100% of Ram-space offsets exceed 2^32, so narrowing is not sound in general — the query is re-expressed exactly here by splitting hi32/lo32, valid only because the window lies in one hi32 bucket, which the probe asserts rather than assumes. No primitive is added and no default changes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Essentials Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe PR adds the ChangesColumn scan benchmark
Estimated code review effort: 3 (Moderate) | ~20 minutes Change: Other Suggested reviewers: ✨ Finishing Touches📝 Generate docstrings
A rabbit reads each line, Comment |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_71156dff-918a-42ac-8e73-7d1b9b9bae2e) |
…eady are `tests/1.98.1` went red on `cargo test --no-run --no-default-features`: that job builds examples too, and `ndarray::simd` is `#[cfg(feature = "std")]`. Cargo.toml documents this exact case three lines above the entry added here — "AMX examples import `ndarray::simd` / `ndarray::hpc`, both `#[cfg(feature = "std")]`, so they must be skipped in `--no-default-features` CI jobs" — and `hex_trie_vs_gemm_probe` / `ternlog_amortization_probe` each carry `required-features = ["std"]`. The new probe did not; that is the whole defect. Reproduced the failing job's own command locally (4x E0432/E0433, "could not find `simd` in `ndarray`"), then re-ran it after the fix: exit 0. The default-featured build and the probe's own run are unchanged, and `cargo fmt --check` is clean. `tests/stable` and `tests/beta` were cancelled by the matrix's fail-fast, not independently red. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv
…ant run, and the wave named as the matrix's own T1 gaps Three read-only censuses (ndarray code, this plan + blackboard, lance-graph consumers) plus verification of every decisive claim before writing it down. Four defects, each verified: 1. Section 14's numbers are stale against the 2026-09-14 (5) STORNO, and the commit that added that STORNO (e5a87e6) DID edit this plan -- one line, section 15's tail law -- leaving every corrected figure in place. So it is not a timing oversight. Section 16.1 tabulates all six corrections; the numbers are not withdrawn, their confidence is. 2. D-GTM-0m names two unrelated probes nine days apart (f1f4023 behavioural soak, d9459f0 hex tenant). Not renumbered -- that would break the commit messages carrying the results -- but no new work may use the id. 3. D-GTM-0n is measured, committed (741e34b) and in no governance doc at all, while section 12.7's own open list says only "0c/0d/0e remain unrun". Its mask-vs-sparse-survivor crossover (0.1-0.8 % active) bears directly on section 12.5 pt 2, which forbids any mask-beats-sparse claim until a sparse arm exists. That question could not be asked while the probe was invisible. 4. The Status header caps at v1.4 while the body runs three sections past it. A fourth hex-tenant run is recorded (section 16.2). It puts the reveal ratio at 161.8x-343.5x, below the previously stated ~200x floor, so across four runs the honest range is ~160-490x. Its ternlogq is 152 ns/pass against the board's 280-300 -- a different host, not a tighter estimate -- and the consequence is worth keeping: coal denominated in "maintained steps" is not a portable unit, because the microseconds fell while the step count rose. What is invariant across all four runs is the shape: the TCAM arm is flat in node size, the range arm tracks it. The wave (section 16.6) is not a new idea. The DuckDB translation matrix already enumerates G6 (mask_set_range), G1 (u8/u16 compare-to-mask) and G2 (ordered u64/i64) as verified-absent with pre-registered falsifiers. N1 = G6: two independent consumers work around its absence and the payoff is measured. N2 = G1: two fixtures now measure the 4x widening cost, and the matrix's falsifier stands unchanged -- if neither moves, the widening was not the cost. N3 = G2: PR #308 answers the matrix's own open question, since its range query needs ordered u64 and 100 % of the real offsets exceed 2^32, so narrowing is unsound. Layering fence recorded for N1: mask_set_range is address-blind. A prefix maps to a contiguous row range only when row order is address order; deciding that is the caller's job, and ndarray must not grow a notion of sortedness. Docs only -- no source, no API, no kernel. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X6y3drwKSE2zSgoexheLFX
… the N3/G2 surface census Two things, both from today's dispatch failure and the N3 preparation it left time for. RULE (new file, .claude/rules/agent-output-durability.md). Two Sonnet workers went out in one message on disjoint files. The one with a single file and a single goal returned with a full report and a landed edit. The one with three parts, four files, twelve functions, a parity group and a correction vanished after 3-5 hours having written NOTHING -- no partial file, no scratch output, no trace. So: every worker tees its progress to its OWN tag-file as it is produced, because a worker that writes only at the end has a single point of failure and takes the whole run with it; and brief size is treated as a reliability parameter, one chunk = one file and one deliverable. The MedCare-rs sibling already carries the same tee discipline for large writes, for the same stated reason. One writer per file -- a shared append-log would re-create at the agent layer the lost-write race the substrate removed at runtime. PLAN section 18, the N3/G2 census, measured by brace-scoped extraction of the impl U64x8 blocks rather than a file-wide grep. That distinction is load-bearing: a file-wide grep finds cmpeq_mask/cmpgt_mask in simd_avx2.rs and simd_scalar.rs, but those hits belong to the U8x64 blocks sitting adjacent in the same files. Read properly, U64x8 has NO compare-to-mask on ANY of the six arms, and U64x2 -- the neon/wasm building block -- has none either, so composing four of those is not available. The consequence decides the brief: N2 was facade-only because U8x64 already carried its primitives everywhere; N3 has no backend primitive anywhere, so both layers are new. On avx512 it is native and cheap (_mm512_cmp*_epu64_mask returns __mmask8, and 8 lanes is exactly one byte), and that is the tier the workspace measures on. On avx2 a per-lane loop is the house pattern and is NOT the same concession as the U8x64 defect: there U64x8 is a scalar polyfill the avx2_int_type! macro generates, holding no __m256i to exploit, and there is no vectorized U64x4-with-compares beside it the way U8x32 sat beside U8x64. Changing what the macro generates is out of N3's scope and is named rather than silently skipped. Also recorded: PR #308 answers the matrix's own open question for G2. It left the gap conditional on how many intended U64-lane predicates are ordered rather than equality; the count is at least one, since find_ram_in_range needs an ordered u64 range and 100 % of the real Ram-space offsets exceed 2^32, making a narrow into the existing i32 family unsound. Docs only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X6y3drwKSE2zSgoexheLFX
blackboard (9): the one fact worth carrying forward is that at `u8` the
packing is FREE -- `U8x64` is 64 lanes, a mask word is 64 bits, so one
`as_chunks::<64>()` group is one whole `out_words[g]` with no shift. That
coincidence is unique to this width and is FALSE for N3's `U64x2 x 4`,
which must pack `out_words[g / 8] |= (bits as u64) << ((g % 8) * 8)`.
Recorded because the temptation in N3 is to assume the u8 shape carries.
Plus the disable table (3 rows, each red-then-green with its observed
assertion), and the chunking evidence: the same class of work that lost
worker B after 3-5 h came back complete as one small scoped chunk -- and
came back with the cross-file `simd.rs` dependency FLAGGED rather than
smuggled into a one-file task, which is what kept six doctests from
silently failing to compile.
CLAUDE.md: the new trap, beside the v4 invocation where it will be read.
The v4 config carries `-D warnings`, so a disable that removes the last
use of a binding turns it into a hard error, the test binary is never
built, and the run emits no `test result:` line at all. Grepped for the
failing assertion that is byte-identical to a guard that is not
load-bearing. It is the workspace's known trap ("a disable that does not
APPLY is indistinguishable from a guard that does not bind") with a second
door -- and the anchor assertion does NOT protect against it, because the
edit genuinely landed; mine reported "3 anchor(s) asserted unique" while
the build was failing. Read the exit status and the `test result:` line,
never only a grep; prefix rather than delete when a disable orphans a
binding.
plan 16.6: N2 marked CODE-LANDED, MEASUREMENT-OPEN. The pre-registered
falsifier -- re-run `hex_tenant_mq_probe`'s 8.9 us re-chain and #308's
`r2il_column_scan_probe` crossover, and if neither moves the 4x widening
was not the cost -- has NOT been run. Existing code is not a moved
measurement, so G1's priority verdict stands unanswered.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X6y3drwKSE2zSgoexheLFX
blackboard (10). Three things worth carrying past this wave: 1. THE ROUTING PROOF, which is the durable method here. A cross-target `cargo check` that passes is evidence about NOTHING unless you know which arm it compiled — and in this tree you cannot infer it, because `simd_neon` and `simd_wasm` are declared behind `#[cfg(feature = "std")]` alone with no target cfg. A wasm check without `+simd128` compiles the SCALAR arm and reports success while never touching `simd_wasm.rs`. So each arm is proved by renaming its `cmpgt_mask` and confirming exactly the target routing to it fails, with the compiler naming the arm back. 2. A census defect in this plan's own table, and it is the MIRROR of the trap the same section documents. 18 warns that a file-wide grep makes an absent surface look present. The nightly arm is a DIRECTORY, not a `simd_<arm>.rs` file, so a census shaped around the single-file arms skipped it and a PRESENT surface looked absent — it has carried 18 compare-to-mask pairs across every width the whole time. Both failures are one error: letting the search SHAPE stand in for the thing searched. It reframes the wave downward in difficulty — N2 and N3 were not adding a capability, they were bringing the stable arms up to a contract the validation arm already stated — and it hands us an unplanned cross-realization differential. 3. The disable restore was `cp` from a backup, not `git checkout`. That sidesteps the known trap (checkout reverts to the last COMMIT, so a disable over uncommitted work deletes it) without needing to commit first, and let the disables run while the last worker was still writing a different file. CLAUDE.md: `CARGO_PROFILE_DEV_DEBUG=0` on every compile, with the measurement that makes it a rule rather than a preference — identical tree, identical run, `target/debug` 1.9 GB with debug info vs 291 MB without, 6.5x, 2319 tests passing either way. The container's writable allowance is a fixed per-session budget that presents as `No space left on device` mid-LINK, so the symptom points at the wrong thing. As ENV, never a committed profile edit; delete `target/debug` before switching rather than growing a second copy beside it. plan 16.6: N3 marked CODE-LANDED, MEASUREMENT-OPEN — same posture as N2. Both pre-registered probes (#308's `r2il_column_scan_probe` re-expressed against the real u64 family; `hex_tenant_mq_probe`'s 8.9 us re-chain) remain unrun, so neither G1 nor G2 has yet been shown to be worth closing. Existing code is not a moved measurement. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X6y3drwKSE2zSgoexheLFX
The plan pre-registered this and left it open through both N2 and N3: "build `gt_u8_to_mask`, re-run both probes; if neither the re-chain nor the #308 crossover moves, the widening was not the cost and G1 drops in priority." The re-chain moved. `hex_tenant_mq_probe` carried its own admission at the widening site -- "widening costs 4x the bandwidth, so `n_gen` below is an UPPER bound on the reveal term" -- and the T1 addition that would replace the bound with a measurement has now landed. This adds the NATIVE u8 arm beside the widened i32 one rather than replacing it, so both are timed in ONE process on ONE dataset. | tier | M1b: 6 masks | | | coal: one re-chain | | | |---|---|---|---|---|---|---| | | widened i32 | native u8 | ratio | widened i32 | native u8 | ratio | | v4 / AVX-512 | 40810 ns | 5064 ns | **8.06x** | 6789 ns | 1006 ns | **6.75x** | | v3 / AVX2 | 43798 ns | 5638 ns | 7.77x | 7189 ns | 1101 ns | 6.53x | In the probe's own cost-model units (v4, x=4) a maneuver goes from **1.02 maintained steps to 0.15** -- a re-chain used to cost a whole maintained step and now costs about a seventh of one. Three things about the method, because the number is only as good as they are: * **Bit-identity is asserted BEFORE either arm is timed.** A timing comparison between two operations that do not produce the same answer measures nothing; both `assert_eq!`s must pass or the probe aborts. * **Same process, same data, same run.** Neither tier reproduces the 8.9 us the plan quotes for this re-chain (v4 6789 ns, v3 7189 ns), so that historical absolute came from a build this one does not reproduce. The RATIO is unaffected by that drift precisely because both arms are measured side by side rather than across runs -- which is why the native arm was ADDED rather than swapped in. * **The widened column's own materialization is OUTSIDE the timed region** (it is built once, up front). So 6.75x is the steady-state sweep cost only, and the conservative reading -- it favours the widened arm. **The ratio is nearly tier-independent (6.75x vs 6.53x), which says the win is a WIDTH effect, not an ISA effect.** Arithmetic that bounds it: for N elements the i32 path issues N/16 compares reading 4N bytes, the u8 path N/64 compares reading N bytes -- 4x fewer instructions AND 4x less memory. Observed 6.5-8x exceeds either alone. The plausible remainder is the packing: the i32 path must shift-and-OR each 16-bit group into its word, while at u8 one chunk IS one whole word and the packing disappears entirely. That attribution is a CONJECTURE consistent with the numbers, not a separate measurement. **Half the falsifier remains unrun, and it is blocked, not skipped.** `r2il_column_scan_probe` is the other pre-registered half (its own header names both gaps: no u8 comparator, no u64 range comparator -- N2 and N3 respectively). It needs a column dump from `r2sleigh-lift`'s `win32_census`, which needs a Win32 PE binary; none exists in this container. Substituting a synthetic dump would produce a number shaped like the falsifier's answer without being it, and that probe's own docs insist on "a real lift rather than a synthetic stream". So G2's measurement stays open. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X6y3drwKSE2zSgoexheLFX
…dependent `masking-ops-state.md` listed two measurements as PENDING and this probe as the instrument for both. Both are now run against the real 12 408-op x86-64 lift. THE BLOCKER WAS STALE, and that is the first finding. The doc read "that probe needs a Win32 PE binary; none exists in this container." One does: r2sleigh/probes/win32-census/legacy_app.exe, 130 613 bytes, committed since 2026-08-27. No synthetic dump was needed. Same shape as the neon-qemu "missing target" that was a missing linker: a report of absence is not evidence of it. G1's second half — ANSWERED. Native eq_u8 vs the widening workaround it replaced: 8.7x on v4, 14x on v3 (34.4x / 26.1x against scalar). The 26-34x is the expected lane count, not an anomaly, and must not be compared to G1's own 6.91x — that pair is u8-vectorized vs i32-vectorized, this one is scalar vs vectorized. Bandwidth corroborates: 52 GB/s L1-resident vs 2 GB/s scalar. G2 — ANSWERED, and TIER-DEPENDENT. The same native find_ram_in_range arm is a 1.51x WIN on AVX-512 and a 0.55x LOSS on AVX2. This file predicted the mechanism (only avx512 epu64 and NEON cmhi have the instruction; avx2/scalar are flat polyfills); the measurement puts a number on it — on v3 the native path is a scalar loop wearing a vector signature and loses to the scalar baseline. Consumers on a v3 floor keep the hi32/lo32 split. Two corrections to PR #308, both of which move a number: - It measured v3/AVX2 while the report implied AVX-512. .cargo/config.toml is x86-64-v3; the host carrying avx512f says nothing about what was compiled. Every arm now runs under both configs and the program prints its own realization line. - No black_box. ndarray's own G1 figures were published 8.06x/6.75x and corrected to 6.91x/5.84x for exactly this. Inputs AND outputs are now protected on every arm, all or none. And the widening tax was real for u8 but NEVER existed for u64: offset read twice is 16 B/op, hi32+lo32 read twice is also 16 B/op. Splitting a u64 into two u32s does not add traffic, it halves the element width — which is why the widened arms beat the native one on v3. #308's crossover number survives; its stated mechanism does not. Unchanged: every arm degrades to <=1.0x above ~200 K ops on both tiers, and the consumer verdict holds — 3 us saved on a binary whose SLEIGH lift costs milliseconds. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv
The arms table promises to describe every validated arm and listed only S/AND/TERN/UNDER, so a reader could not reconstruct the five-arm methodology or its pass counts from the docs. Adds NATIVE with its three predicate passes plus the mask combination (4 total). Two things the finding did not name, fixed with it because they are the same staleness: - The gap section still read "two primitive gaps this probe had to work around" in the present tense, and closed on "a general range needs the primitive" — both contradicted by the Measured section a few lines below, which reports the primitives shipped and measured. Re-framed as what the FIRST run found (the widened arms still exist and still need their rationale) with the closure stated inline. - The other four rows did not say their columns are WIDENED, which is the whole reason NATIVE is a different arm rather than a faster spelling of the same one. Named in each row. Gates: clippy -D warnings (v4 config), cargo fmt --check, and `cargo test --no-run --no-default-features` — the job that went red on #308 for a missing required-features gate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv
Answers the question
r2il::columnsasks and declines to answer itself. Its module doc names this crate's mask surface (eq_u32_to_mask,masked_strided_group_sum), lays the columns out for it, takes nondarraydependency on purpose — and says plainly: "Whether that is worth doing is a profiling question nobody has answered."Measured, consumer-side, on a real x86-64 lift: 12 408 p-code ops from r2sleigh's committed
win32-censusfixture (PE32+,.text7 688 B), not a synthetic stream.Method
Four arms compute an identical mask for the census's own IAT/prefetch query —
space == Ram && lo <= offset < hi— with a bit-for-bit equivalence gate before any timing, swept 256 → 3.2 M ops:Su8/u64columns, no wideningAND*_to_mask+ 3mask_and_assignTERN*_to_mask+mask_ternlog::<AND3>+mask_and_assignUNDER_underchain — each predicate narrows the live maskSis the honest baseline because it needs no widened columns: charging the mask arms for the layout they require is the comparison a consumer actually faces.Result (Xeon @ 2.10 GHz, avx512f/bw/vl, release, 3 runs, ns/op)
Spans past 12 408 are the real stream tiled — labelled
T, and evidence about throughput only, never about program shape.Four findings
Sreads 9 B/op, the mask arms read 12 B/op of widened columns and write four mask buffers. The layout's own motivation — fewer bytes touched — is partly spent paying for the primitives' value types.TERNandANDare within noise at every span. The cost is the four passes over value columns, not the three combines fusion removes. Fusion pays where a caller already holds the masks._underis the arm that survives scale — no separate combine, no extra buffers: best at the real size (1.42×) and the only one near parity at 3.2 M.Two primitive gaps, worked around and recorded rather than closed
u8comparator.OpColumns::{tag,space}areVec<u8>; the facade's narrowest value type isu32, so a consumer keeps a widened copy at 4× the bytes of the column it scans.u64range comparator.ternary_match_u64_to_maskis exact-with-don't-care; the ordered family stops ati32. Measured, 100 % of Ram-space offsets exceed 2³² (image-based,0x1_4000_105e…0x1_4000_8398), so narrowing is not sound in general. The query is re-expressed exactly by splittinghi32/lo32— valid only because the window lies inside onehi32bucket, which the probe asserts rather than assumes.Whether to add
ge/lt_u64_to_maskis a decision, not a drive-by: this PR adds no primitive, changes no default, and touches nothing outsideexamples/.Pairs with AdaWorldAPI/r2sleigh#14 (the env-gated column dump that feeds it).
🤖 Generated with Claude Code
https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv
Summary by CodeRabbit