Skip to content

Add runtime debug logs across plan publication and query execution - #756

Open
zzylol wants to merge 51 commits into
issue-752from
issue-755
Open

zzylol wants to merge 51 commits into
issue-752from
issue-755

Conversation

@zzylol

@zzylol zzylol commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Problem and behavior

Before this PR: following a request across planning, deployment, shared execution and SDS required unrelated logs with incomplete node and state identities.

After this PR: optional asap_runtime_debug events identify the actual stage: Planner selection, backend candidate admission/cost selection, deployment binding/activation, shared DAG execution, SDS publication, bound lookup/validation, and recovery. Candidate events include status, cost, rejection reason and search coverage. State events distinguish stored_output_id from definition_id; execution events identify dependencies and inclusive node duration. Failed bound reads use debug-level error events.

Use RUST_LOG=info,asap_runtime_debug=debug in both processes. Call IDs are process-local. The architecture/stage table and correlation limits are documented in docs/runtime-debugging.md. Diagnostics describe the implemented path; they do not certify exhaustive physical-candidate generation or change selection behavior.

Stack

Based on #761 to instrument its current runtime-evidence costing path. #756 and #766 are independent follow-ups; correctness/evaluation PRs do not depend on this diagnostics PR. Shared operators come from ASAPPlanner #462.

Validation

Strict all-target Clippy passes on the current merged branch. The #761 base passes 120 type, 448 control-plane and 896 data-plane unit tests, nine native candidate tests, and six real-process Rate aggregation/restart tests. Those base checks validate execution semantics; the diagnostic spans report stages without changing them.

Native maintenance, stored-summary binding and bound-native lookup now have distinct spans. Documentation covers grouped Sum and CMS/CountSketch heaps, both placements, and overlapping complete windows. Logs expose selected execution, not an exhaustive-search or performance claim.

Closes #755.

Comment thread control_plane/src/main.rs Outdated
@milindsrivastava1997

milindsrivastava1997 commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

@zzylol

I ddded the following:

  • Operator label on DAG node events (op, e.g. summary_estimate, logical/temporal): previously node events only had node_id, which meant cross-referencing the compiled plan to know which operator failed.
  • Per-leaf failure logs in exact-leaf preparation and catalog validation (node_id + op): most capability misses happen here, before DAG evaluation, and were only visible as a call-level error with no node context.

@zzylol

zzylol commented Sep 22, 2026

Copy link
Copy Markdown
Contributor Author

@milindsrivastava1997 Thanks for the logging suggestion. I made the controller and data plane consistent: both now use only RUST_LOG (default info), and the data plane --log-level flag is removed. For workflow traces, set RUST_LOG=info,asap_runtime_debug=debug on both processes.

I also expanded the installed query DAG events so every evaluated node reports its IR type (op), bounded syntax details (syntax), and input node IDs (inputs). For example, the aggregation E2E run shows logical/aggregate/sum with grouping=by(service), exact_readout/count, and logical/binary with operation=Div. The real success and failure logs are in the PR description; the focused aggregation E2E test and node-label tests pass. Full query text is omitted from the new syntax field.

@zzylol
zzylol changed the base branch from refactor/query-plan-dag-execution to issue-752 September 27, 2026 12:49
# Conflicts:
#	control_plane/src/physical/compiler.rs
# Conflicts:
#	crates/asap_types/src/query_plan/native.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Add debugging logs for each components following the workflow of control plane and data plane for debugging

2 participants