Skip to content

Pass 1: single-measure filtered aggregates - #608

Draft
zzylol wants to merge 1 commit into
stack/509-y2-filtered-aggregatesfrom
stack/509-y2b-filtered-pass1
Draft

zzylol wants to merge 1 commit into
stack/509-y2-filtered-aggregatesfrom
stack/509-y2b-filtered-pass1

Conversation

@zzylol

@zzylol zzylol commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Stack: Wave 3 chain: #605 → #607 → #608 → #609 → #610

Rebased on main d4869a7 (DF 54).

Stack: … → #605 → #607 → this PR

Why

Part of #509 and #580 (J, filtered aggregates); Q47 (a). Pass 1 listed alternatives for a filtered single-measure aggregate, but realize rejected every non-pass-through choice ("filtered or HAVING aggregate"). So these alternatives failed in Stage 1, and the query could only run as written, which #607 makes possible. This covers SQL FILTER (WHERE …) and COUNT(y) over a nullable y, which is lowered as a count filtered by y IS NOT NULL.

What

SELECT g, COUNT(*) FILTER (WHERE x > 0) AS c FROM events GROUP BY g at ε = 0.1, δ = 0.01:

alternatives that compose
Before this PR pass-through only. Count acc, CMS, CountSketch, UnivMon and HydraCms fail: unsupported local realization: filtered or HAVING aggregate
After this PR all 6. Each SummaryAgg carries filter: x > 0. An alternative binds in the executor exactly when its unfiltered counterpart does. The Count accumulator and HydraCms execute to a 2, b 0, c 1, matching the exact plan.

For a filtered SUM and a filtered approx_percentile_cont, the exact Sum accumulator, KLL and DDSketch execute to the exact answer. They read NULL for the group with no matching row.

Selection: the Examples 1, 3a, 3b and 4a stage documents are byte-identical to #607's, and the committed Example 1 fixture is unchanged. Example 2 has no filtered measure (planner_layering_example2 asserts this), and its tests are unchanged. I did not tune costs.

How

  • Pass 1 (logical_candidates::realize): a single measure's filter becomes SummaryAgg.filter, which reads the same input rows. This applies to every family, Hydra, and tumbling panes, since each pane carries the filter. HAVING stays rejected. The SQL frontend lowers it to a Filter above the aggregate anyway. A whole-expression (absorbing) target is never filtered.

  • Coverage stays whole-source (Check declared summary coverage population against subtree filters #570: the declared population is trusted). A filtered state holds a subset of its source, so whole-source over-states its population. That is the conservative direction:

    • merges need proven disjointness, and a larger claim proves less;
    • CSE compares the operator, filter included, so a filtered state is never shared with an unfiltered one.

    A column = literal filter could be declared as that population, but nothing reads populations from Pass 1 yet. I did not do this.

  • Types: FinalizeExactAccumulator and a quantile SummaryEstimate over a filtered build are declared nullable, like the filtered aggregate each realizes. Without this, a filtered exact SUM read 0.0 for an empty group.

Tests

These failed or did not exist on #607:

  • filtered_aggregates::pass1_offers_filtered_count_alternatives: the six alternatives, the filter on each, bind parity with the unfiltered plan, and execution matching the exact plan.
  • filtered_aggregates::pass1_filtered_sum_and_quantile_alternatives_execute: SUM (exact acc) and percentile (KLL, DDSketch within 2%), with NULL for the empty group.
  • stage_pipeline_selection::sql_filtered_aggregates_dp_equals_exhaustive: a filtered grouped count × a filtered percentile, 18 combinations. The DP equals exhaustive, and none is rejected while being built.

filtered_aggregates::exact_filtered_sum_is_null_for_groups_without_matches from #607 now selects the filtered Sum accumulator, and it still reads NULL.

Not in this PR

  • Multi-measure split: an aggregate with several measures, filtered or not, stays a single pass-through target. Splitting it into single-measure targets, each with its own filter, is a follow-up.
  • HAVING on Aggregate stays rejected. The frontends never emit it; SQL HAVING is a post-aggregate Filter.
  • Gaps that predate this PR and that filters do not change: a per-group CMS/CountSketch for SQL COUNT(*) does not bind (keyed summary weight must be a finalized value column), and filtered ingestion-time precompute is unsupported (executor: filtered aggregates and summary builds #607).

Gates

Refs #509, #580, #605.

🤖 Generated with Claude Code

Pass 1 listed alternatives for a filtered single-measure aggregate (SQL
FILTER, or the IS NOT NULL a COUNT(nullable) implies) but every
non-pass-through choice failed to compose. The measure's filter now
becomes SummaryAgg.filter, so exact accumulators and sketches (Hydra
included) compose, build, and execute. Coverage stays whole-source.

A FinalizeExactAccumulator or quantile SummaryEstimate over a filtered
build is declared nullable, as the filtered aggregate it realizes is.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol
zzylol force-pushed the stack/509-y2-filtered-aggregates branch from 4c68af1 to e1a4e6d Compare October 5, 2026 06:21
@zzylol
zzylol force-pushed the stack/509-y2b-filtered-pass1 branch from cabe9d9 to a90e186 Compare October 5, 2026 06:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant