Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
f489f27
test: isolate observed workload statistics contracts
zzylol Sep 28, 2026
235ec88
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
f7fb821
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
d53d561
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
4c76e04
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
8e5de8d
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
9854bf4
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
ef0cb7b
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
0891697
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
b0b95bf
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
57e38f4
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
21c884a
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
9441b2a
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
339203e
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
7513753
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
d7bfe7d
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
1d5e4ae
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
fb9f0ef
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
f14244e
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
ee29e5d
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
1260c45
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
4f8f7bd
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
a7955ca
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
896af88
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
28c7698
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
b854ebe
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
fdb3d69
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
9267aef
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
217ebcb
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
5679d06
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
5fbdfb6
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
b45a7bb
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
32a4089
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
ec74f11
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
bd3ec32
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
362cc20
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
43d1447
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
7f514e3
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
2c3334f
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
34ccdea
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
b1ce589
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
c780bd2
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
4de07bd
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
e7bcf74
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
666c366
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
2357d89
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
e2a515e
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
a1bb8df
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
e06a36b
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
0ddbd7a
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
7b4ea99
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
346219f
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
08e6c5f
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
4e09c61
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
e9c8269
Merge branch 'test/physical-execution-e2e' into test/workload-statistics
zzylol Sep 28, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
99 changes: 99 additions & 0 deletions control_plane/tests/workload_statistics.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
//! Statistics-consumer contracts. Controlled observations are not live telemetry.
use control_plane::physical::{compiler::BackendLocalPlanningInput, erp::ErpShapeObserver};
use serde_json::{json, Value};

fn wire() -> Value {
let mut wire: Value = serde_json::from_str(include_str!(
"../../docs/examples/asapquery-planning-snapshot.json"
))
.unwrap();
wire["query_workload"]["repeating_queries"][0]["query"] = json!("sum_over_time(m[1m])");
wire["data_workload"]["ingestion_rate"] = json!({
"value":25.0,"source":"observed","observed_at_ms":9500,"valid_for_ms":1000
});
wire["data_workload"]["input_cardinality"] = json!({
"value":125,"source":"observed","observed_at_ms":9500,"valid_for_ms":1000
});
wire
}

/// Source rate, source cardinality and query cadence remain distinct quantities.
#[test]
fn observed_workload_facts_survive_binding_without_changing_units() {
let mut wire = wire();
for interval in [5000, 20000] {
wire["query_workload"]["repeating_queries"][0]["demand"] =
json!({"fixed_interval_at":{"interval":interval,"evaluation_phase":0}});
let input: BackendLocalPlanningInput = serde_json::from_value(wire.clone()).unwrap();
let expected = input.data_workload.clone();
let (request, _) = input.into_physical_compilation_request().unwrap();
assert_eq!(request.data_workload.as_ref(), Some(&expected));
assert_eq!(
request.queries[0]
.summary_lifecycle_inputs
.ingestion_rate_per_second,
25.0
);
assert_eq!(
request.queries[0]
.summary_lifecycle_inputs
.evaluation_interval_ms,
interval
);
assert_eq!(expected.input_cardinality.value, Some(125));
}
}

/// Missing, expired and future observations cannot be priced as zero ingestion.
#[test]
fn unavailable_rate_observations_are_rejected() {
for observation in [
json!({"value":null,"source":"unknown","observed_at_ms":null,"valid_for_ms":null}),
json!({"value":25.0,"source":"observed","observed_at_ms":8000,"valid_for_ms":1000}),
json!({"value":25.0,"source":"observed","observed_at_ms":11000,"valid_for_ms":1000}),
] {
let mut wire = wire();
wire["data_workload"]["ingestion_rate"] = observation.clone();
let input: BackendLocalPlanningInput = serde_json::from_value(wire).unwrap();
assert!(
input.into_physical_compilation_request().is_err(),
"{observation}"
);
}
}

/// No recent arrivals does not imply no retained series.
#[test]
fn observed_zero_rate_does_not_erase_cardinality() {
let mut wire = wire();
wire["data_workload"]["ingestion_rate"]["value"] = json!(0.0);
let input: BackendLocalPlanningInput = serde_json::from_value(wire).unwrap();
let (request, _) = input.into_physical_compilation_request().unwrap();
assert_eq!(
request.data_workload.unwrap().input_cardinality.value,
Some(125)
);
assert_eq!(
request.queries[0]
.summary_lifecycle_inputs
.ingestion_rate_per_second,
0.0
);
}

/// Repeated events count toward workload, not distinct cardinality; overflow invalidates the window.
#[test]
fn observation_population_counts_events_and_distinct_keys_separately() {
let mut observer = ErpShapeObserver::with_limits(2, 4).unwrap();
for _ in 0..30 {
observer.observe("series-a", 0).unwrap();
}
for _ in 0..10 {
observer.observe("series-b", 1).unwrap();
}
let observation = observer.snapshot().unwrap();
assert_eq!(observation.observation.observed_events, 40);
assert_eq!(observation.observation.cardinality, 2);
assert!(observer.observe("series-c", 2).is_err());
assert!(observer.snapshot().is_none());
}
17 changes: 17 additions & 0 deletions docs/design_docs/workload-statistics-validation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Workload statistics validation

This test PR isolates the consumer contract from ranking and sketch accuracy.
`control_plane/tests/workload_statistics.rs` verifies units, retained-series
cardinality with zero arrivals, expired/future/missing observations, and bounded
population accounting. Its controlled observations are not live telemetry.

The intended measurement sources remain remote_write accepted-sample counters,
query-tracker executions, and Prometheus series observations. Rates need explicit
observation windows and counter-reset handling; series scope must match the input
computation. Query cadence and ingestion cadence are different quantities.

The existing discovery replay adapter reports a derived finite-replay rate and
declared query recurrence. Those are not live ingestion/query measurements. This
PR does not claim the three production collectors are fully wired together.
Level 3 must preserve original trace timing, observation provenance and scope;
missing observations make the real-evidence run incomplete.
Loading