Skip to content

feat: compile multi-selector PromQL subtrees and more range functions - #487

Open
zzylol wants to merge 4 commits into
feat/physical-compile-promql-fallbackfrom
feat/physical-compile-coverage-3
Open

zzylol wants to merge 4 commits into
feat/physical-compile-promql-fallbackfrom
feat/physical-compile-coverage-3

Conversation

@zzylol

@zzylol zzylol commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #486.

Rebuilt into the linear stack on main. New integrate: commit(s) fold in integration-branch resolutions this PR needs on top of the earlier stack: "extend asap-types series identity with #486/#487 shapes". Conflict resolutions are recorded in the messages of: "feat(physical): compile multi-selector PromQL fallbacks with matching, without and @".

Why

#486 compiles a retained PromQL subtree (Fallback) only when it reads one selector and uses a small set of range functions. Common backend queries such as rate(a[5m]) / rate(b[5m]), sum without (instance) (m), irate(m[5m]) and m @ 1700000000 still go through the backend's own lowering (coverage rows 28 and 30).

What

  • Each selector of a Fallback gets its own raw-series input slot, promql_fallback::raw_series_input(node, i). The lowering is now a small operator DAG.
  • Vector-vector arithmetic uses PromQL one-to-one matching, with on(...) and ignoring(...). There are two new operators. series_labels rewrites a row's label set, either the series identity or the plain label columns. series_binary matches equal label sets. A duplicate match group is an error, as in Prometheus. The result drops __name__.
  • without aggregation reuses series_labels, so the identity drops the excluded labels and __name__.
  • New range functions: irate, idelta, changes, resets, last_over_time, and exact quantile_over_time.
  • @ <timestamp> works on selectors and subqueries. @ start() and @ end() are rejected because they depend on the range query.
  • The coverage doc records the new coverage. It also states exactly what the IR must carry for histogram_quantile, which is not implemented here.

Before this PR

compile(Fallback{ a / on(job) b })        -> Err("operator has no dynamic series-identity realization")
compile(Fallback{ sum without (inst) (a) }) -> Err("dynamic without grouping requires label-set projection")
compile(Fallback{ irate(m[5m]) })          -> Err("unsupported PromQL range function")
compile(Fallback{ m @ 120 })               -> Err("... @ is unsupported")

After this PR

With a{job=x,inst=1} = 10 and b{job=x,inst=2} = 4:

a - b             -> {}              (label sets differ)
a - on(job) b     -> {job="x"} 6
a - ignoring(inst) b -> {job="x"} 6

With a{job=x,inst=1}=1, a{job=x,inst=2}=2, a{job=y,inst=1}=4, sum without (inst) (a) returns {job="x"} 3, {job="y"} 4. Each case matches a hand-computed Prometheus result in tests/promql_fallback.rs.

Remaining

  • histogram_quantile: the IR must carry the le column and without (le) grouping. The doc lists the full requirements.
  • Comparisons and set operators, group_left and group_right, and non-literal scalar operands.
  • @ start() and @ end(), other subquery operands, and other functions.
  • The rest is listed in the coverage doc.

Validation

  • cargo fmt --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test --workspace --no-fail-fast

An independent reviewer agent, which did not write the code, reviewed the diff. Its three findings are fixed, with regression tests:

  • An empty identity label value is now treated as absent.
  • An empty side now returns an empty result before the duplicate check.
  • A non-literal scalar operand is now rejected at realization.

Two behaviors are deliberately unchanged. Results other than vector matching and without keep __name__, following #486's adapter contract. NaN q does not survive a JSON round-trip, which is an existing limitation.

🤖 Generated with Claude Code

zzylol added a commit that referenced this pull request Sep 30, 2026
#477 moved series-identity resolution into asap-types; #486 and #487 extended
the physical-layer copy. Port the extended rules into asap-types and keep
promql_rows delegating.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol and others added 4 commits September 30, 2026 18:09
…tile over time

Per-series windows evaluate these PromQL range functions with Prometheus
semantics over fresh (non-stale) samples.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, without and @

Each Fallback selector reads its own raw-series slot. Vector-vector
arithmetic uses PromQL one-to-one matching with on/ignoring via new
series_labels and series_binary operators; without grouping rewrites the
series identity; @ <timestamp> fixes selector and subquery evaluation.

Conflicts with earlier stack changes resolved to the integration tree:
- crates/asap-physical-operators/src/operators/mod.rs: 9a13ae4 integrate: extend asap-types series identity with #486/#487 shapes
- crates/asap-physical-operators/src/physical_planner/promql_rows.rs: 9a13ae4 integrate: extend asap-types series identity with #486/#487 shapes

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…antile IR needs

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#477 moved PromQL series-identity resolution into asap-types. The
fallback shapes compiled by #486 and #487 (time shifts, subqueries,
scalar bridges, and arithmetic between series) also need identity
realization there. Taken from integration commit 9a13ae4.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@zzylol
zzylol force-pushed the feat/physical-compile-promql-fallback branch from a4d7e16 to fd0bb0a Compare September 30, 2026 18:27
@zzylol
zzylol force-pushed the feat/physical-compile-coverage-3 branch from eb34e4f to d4375b3 Compare September 30, 2026 18:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant