Skip to content

feat: compile Planner selections into physical DAG candidates - #475

Open
zzylol wants to merge 1 commit into
split/462-d-runtimefrom
split/462-f-physical-planner
Open

zzylol wants to merge 1 commit into
split/462-d-runtimefrom
split/462-f-physical-planner

Conversation

@zzylol

@zzylol zzylol commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Part 4 of the #462 split. Stacked on #474; it builds on #472, which is already merged into main.

Before this PR

The Planner selects logical Post-ASAP candidates (#472, merged) and has a shared runtime (#474), but nothing turns a selection into executable physical DAGs. Deployments do that lowering themselves.

After this PR

Logical Post-ASAP candidate + maintenance requirements
    → physical_planner compilation
    → precompute/query physical DAGs + typed materialization frontier
    → deployment: source/state binding, costing, storage
    → shared runtime (#474)
  • Reader-independent compilation of selected candidates. Binding never repeats logical lowering.
  • Bounded frontier enumeration and comparable workload-cost selection, including Rate → grouped Sum at both placements.
  • Automatic temporal KLL pane compilation, and PromQL row/value lowering with dynamic-label vectors.
  • The compiler emits typed readouts (feat: add native physical operators and shared DAG runtime #474): logical SketchQuery passes through unchanged, so there is no Statistic/string-parameter round trip.
  • CompiledPhysicalDag and PhysicalCandidate derive serde and are validated on deserialization. They have no Planner byte format or version.
  • dag is kept as a compatibility re-export.

Boundary change from #462

The Planner stays deployment-agnostic. These parts of #462 are left out and go to ASAPQuery-backend in a separate PR:

  • the stored-definition semantic fragment and its dataset binding;
  • SDS wire decoding and delta reconstruction, plus native batch frames;
  • the versioned JSON envelopes for physical plans.

The design doc states this ownership.

Validation

cargo test --workspace passed 1356 tests, covering physical DAGs, plan recovery, precompute candidates and populations, PromQL values and binaries, weighted TopK binding, current-series heaps, and the KLL pane, SQL→physical and lifecycle end-to-end tests. Stored pane states are passed in memory; byte encodings are tested in sketchlib and deployments. cargo clippy --workspace --all-targets -D warnings is clean. These are shared-library tests, not deployment end-to-end tests.

Split stack: #472 · #473 · #474 · this PR.

🤖 Generated with Claude Code

Add `physical_planner`: reader-independent compilation of selected logical
candidates into precompute/query physical DAGs with typed materialization
frontiers, bounded frontier enumeration, workload cost selection, temporal
KLL pane compilation and PromQL row/value lowering. `dag` remains a
compatibility re-export.

Tests cover physical DAGs, plan recovery, precompute candidates and
populations, PromQL values and binaries, weighted TopK binding and
current-series heaps, plus Planner-to-execution integration tests. The design
doc describes the ownership boundary with deployments.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant