Conversation
This was referenced Oct 4, 2026
zzylol
force-pushed
the
stack/509-w7b-1-frequency-runtime
branch
from
October 4, 2026 18:25
502e276 to
bceb13e
Compare
This was referenced Oct 4, 2026
Draft
zzylol
changed the base branch from
stack/509-e2-keyed-additive-topk
to
stack/509-w7a-univmon-accuracy
October 4, 2026 18:26
…ntities Port of the parked runtime PRs for #509 Example 2 onto asap-executor: - native UnivMon build and Cardinality / FrequencyL2 / FrequencyEntropy readouts (the old stack's 14c8ac8, a prerequisite missing here); - #557: a UnivMon build accepts Utf8, Int64 and Bool identities as type-tagged keys; heap key bytes count toward state memory; - #559: exact FrequencyL2 / FrequencyEntropy reducers (bits, NULL skipped, 0 for an empty population, memory-accounted, cooperative); - #563: exact Cardinality reducer over typed tuples. The legacy raw-cost adapter lists the three intents as hash aggregates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol
force-pushed
the
stack/509-w7b-1-frequency-runtime
branch
from
October 5, 2026 06:21
bceb13e to
c4cd1d0
Compare
zzylol
force-pushed
the
stack/509-w7a-univmon-accuracy
branch
from
October 5, 2026 06:21
aab5e7e to
a6e5cda
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebased on main d4869a7 (DF 54).
Wave 1 chain: #594 → #593 → #592 → #591 → #595 → #596 → #597 → #598 → #599 (on #589)
Why
#509 Example 2 plans one UnivMon over
src_ipforCOUNT(DISTINCT src_ip), the entropy of thesrc_ipdistribution and the L2 norm of per-src_ipcounts. Pass 1 also keeps an exact candidate for each statistic. On the current stack none of these runs natively:summary family has no native DAG state implementation), and a summary build accepts only Float64 updates;Cardinality,FrequencyL2andFrequencyEntropyaggregates fail to bind (aggregate intent has no native implementation).The parked draft PRs #557, #559 and #563 fixed this on the old stack (
asap-physical-operators). This PR ports them ontocrates/executor. Part of #509 / #580 (W8).What
Operator::summary_build(flows(src_ip: Utf8), UnivMon, ..)→Err("summary numeric update requires Float64")Aggregate [Cardinality / FrequencyL2 / FrequencyEntropy {accuracy: Exact}]→ no native implementationSELECT COUNT(DISTINCT src_ip) FROM flows WHERE keepcannot execute0/0/2on the test inputsHow
validate_native_family,validate_sketch_evaluationandvalidate_stateaccept UnivMon with bare count, Cardinality, FrequencyL2 and FrequencyEntropy readouts.AccumulatorUpdater::update_value(&Value, ts)defaults to the Float64 contract. UnivMon inserts Bool/Int64/Utf8 as type-taggedValue::key()bytes, so Int64 keys above 2^53 stay distinct. Heap key bytes count toward state memory.Reduction::FrequencyL2/FrequencyEntropy(col): NULL skipped, non-finite floats rejected,0.0for an empty population, entropy in bits, per-key memory reservation and cooperative checkpoints.Reduction::Cardinality(cols): a tuple with any NULL is skipped; the grouped key encoding keeps tuple boundaries and folds signed zero and NaN payloads.plan-selection/src/cost/query_physical_lowering.rs) lists the three intents as hash aggregates, as feat(runtime): execute exact frequency L2 and entropy #559 and feat(runtime): execute exact distinct identity counts #563 did.plan_stagesalready prices any aggregate.Name and path changes only:
asap_physical_operators→asap_executor,pre_asap/post_asap→types::ir::{operator, schema, properties},PostAsapDAGNode→PhysicalASAPDAGNode. The test helperexecute_raw_rows(from #563) usescompile_physical_asap_dagandroots[0]. The developer-doc notes from #557 and #559 are not ported.Tests
Carried over from #557 (
univmon_execution::typed_frequency_keys_preserve_identity, two kernel tests), #559 (exact_frequency_*, the memory-budget test) and #563 (exact_cardinality_*,sql_cardinality), plus 14c8ac8'sone_univmon_state_answers_distinct_l2_and_entropy.precompute_raw_samplesnow expects UnivMon to compile and checks its three readouts (the old stack's d206f44), because UnivMon has native state. Before the port, both UnivMon tests fail at runtime and the reducer tests do not compile (no variant FrequencyL2/Cardinality).Gate:
cargo fmt --all --checkandcargo clippy --workspace --all-targets --all-features -- -D warningspass.cargo test --workspace: 1,527 passed, 10 ignored (#589: 1,517 / 10).Ports #557, #559, #563. Next: SQL recognition (#598).
🤖 Generated with Claude Code