Conversation
zzylol
force-pushed
the
feat/lifecycle-candidates
branch
from
September 30, 2026 02:14
59b41bd to
4bb8c86
Compare
zzylol
force-pushed
the
feat/compile-once-cuts
branch
from
September 30, 2026 02:14
781d02b to
215f4d7
Compare
`compile_candidate` re-lowered the whole Post-ASAP DAG for every materialization frontier, and `enumerate_frontiers` compiled it once more. Number helper operators from their Planner node (`u64::MAX - node_id`, at most one helper per node) so every boundary choice is a subgraph of one lowering. `cut_candidate` partitions a `compile` result for one frontier and `enumerate_compiled_frontiers` enumerates over it; both produce candidates byte-identical to per-frontier recompilation. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The #462 split no longer exposes PhysicalCandidate::encode; its serde form gives the same byte-for-byte comparison. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The lifecycle layer assigns each node's timing; physical compilation now reads it. `frontier_from_timing` returns the ingestion-time nodes read by query-time nodes (or an ingestion-time root) and rejects a query-time node feeding an ingestion-time one, so each lifecycle assignment is a `cut_candidate` of one `compile` result. `enumerate_compiled_frontiers` is private: placement comes from timing, and its only caller is `enumerate_frontiers`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For the KLL quantile and grouped Rate->Sum fixtures, ContinuouslyMaintained and Ephemeral timed DAGs cut one compilation into exactly the candidates `compile_candidate` builds. The hand-written timing frontier in the chosen lifecycle test now uses `frontier_from_timing`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zzylol
force-pushed
the
feat/compile-once-cuts
branch
from
September 30, 2026 02:48
215f4d7 to
abb0a85
Compare
zzylol
changed the base branch from
feat/lifecycle-candidates
to
feat/lifecycle-timing
September 30, 2026 02:48
Number helpers as u64::MAX - (node << 16) - index so a node that lowers to an operator chain keeps deterministic, traversal-independent helper IDs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With several retained states, only those read by a query-time node or forming the root are cut points. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #482 (on #476, on #475).
Why
The lifecycle layer decides timing (#482), and physical compilation reads it. A
deployment should compile each query DAG once and turn each lifecycle
assignment into a cut of that compilation. Right now
compile_candidatere-lowers the whole Post-ASAP DAG for every frontier, and callers work out the
frontier from timing themselves (#482's e2e test does this in a hand-written
helper).
What
u64::MAX - (node_id << 16) - index,at most one helper per node) instead of from a running counter. Every
frontier then gives a subgraph of the same lowering, so a cut needs no
renumbering.
cut_candidate(&compiled, frontier)splits onecompileresult into theprecompute and query DAGs of a
PhysicalCandidate. Nothing is lowered again.frontier_from_timing(&PostAsapDag) -> Result<Vec<NodeId>>finds thefrontier in a timed DAG from
execution_timed_dag: the ingestion-time nodesthat query-time nodes read, plus the root if it runs at ingestion time. It
rejects a DAG where a query-time node feeds an ingestion-time node. It is
public because the deployment owns the compile/cut loop and needs this
mapping. Without it, every caller writes its own copy of the rule.
enumerate_compiled_frontiersis no longer public. Placement now comes fromlifecycle timing, so the only callers were
enumerate_frontiersand tests.compile_candidateslowers once and cuts once per frontier.frontier_from_timingreplaces the hand-written frontierhelper.
Before this PR
After this PR
For
quantile(0.99, latency)andsum by(job)(rate(m[1m])), theContinuouslyMaintained frontier is the retained state and the Ephemeral
frontier is empty. Both cuts come from one compilation.
Compatibility
u64::MAX - (node_id << 16) - index. Encoded physical DAGs thatcontain helpers are different byte for byte from before, so fixtures or
quotes that embed them must be regenerated.
Binarylowers differently from a query-time one. For thatnode, the timing at compile time must match the assignment being cut. This
is documented on
frontier_from_timing.Validation
summary_maintenance_lifecycle_e2e::lifecycle_timing_cuts_one_compilation:covers the KLL quantile fixture and grouped Rate→Sum. One compilation of the
Ephemeral-timed DAG is cut by both the ContinuouslyMaintained and the
Ephemeral timed DAGs. Each cut serializes byte for byte like
compile_candidateon that timed DAG, and each frontier is the expected one.physical_planner::candidates: across two timings, eachPlanner node is lowered exactly once (counted) and both cuts match
compile_candidate. An ingestion-time root is the frontier. A query-time nodefeeding an ingestion-time node is rejected.
precompute_candidates.rs: every enumerated frontier cuts to the same bytesas per-frontier recompilation. The top-k
Sorthelper isu64::MAX - rootand is rejected as a boundary.
cargo fmt --checkandcargo clippy --workspace --all-targets -- -D warningspass.
cargo test --workspace --no-fail-fast: 1378 passed, 0 failed.🤖 Generated with Claude Code