test(promql-compliance): split quantile suite, drop sparse-cadence fixtures - #737
Merged
milindsrivastava1997 merged 1 commit intoSep 23, 2026
Merged
Conversation
…xtures - Move quantile queries to a new quantiles suite on a dense (1s) and wide (120-series) dataset so nearest-rank vs interpolated quantiles differ by less than the 1% tolerance. - Give each aggregations series a distinct scrape interval (60s..10s) so topk(count_over_time) has no ties. - Remove the sparse-cadence aggregations fixture and sparse-checkout; the dense dataset becomes aggregations.yaml and host=b moves to single-rate. - Add new aggregation query shapes (topk/sum over rate, quantile ratios). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
milindsrivastava1997
deleted the
test/promql-compliance-quantile-dataset
branch
September 23, 2026 18:07
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
New
quantilesdataset and suite. ASAPQuery's KLL quantile returns an observed sample (nearest rank), while Prometheus interpolates between neighbours. On 5-sample windows or 6 series, that alone gave 6–15% differences. The new dataset keeps the neighbour spacing small:quantile_dense: 1s cadence, 300 samples per 5m window, below the default KLL K=500.quantile_wide: 120 series (3 jobs × 40 instances).A local simulation of both quantile rules gives a worst-case error of 0.5%, under the 1% tolerance, while p10 and p90 stay 17–25% apart, so a query that computed the wrong quantile still fails. All 31 quantile queries moved here from the
aggregationssuite.Per-series scrape intervals in
aggregations.yaml(60/30/20/15/12/10s).count_over_timeis now distinct per series, sotopk(count_over_time)no longer picks different tied series on each engine.Removed sparse fixtures. For now we assume one data point per scrape interval. The sparse-cadence
aggregations.yamlis deleted and the dense variant takes its name.sparse-checkout.yamlis deleted: itscheckout_upseries were never queried, and itshost=bseries moved tosingle-rate.yaml.run-allnow has 3 cases. The fixture validation test now checks bothaggregationsandquantiles(dataset and suite).make run-allresultssingle-rate-temporalquantilesaggregationsThe 5
aggregationsfailures are all aggregations overrate. ASAPQuery returnsbad_data: No result for queryfor every instant and range evaluation of them, while plainrate(data[5m])passes:topk(3, rate(data[5m]))topk by (job) (3, rate(data[5m]))topk by (job, instance) (3, rate(data[5m]))sum by (job) (rate(data[5m]))sum by (job, instance) (rate(data[5m]))These look like an engine or planner gap, not a test issue. They will be tracked separately.
🤖 Generated with Claude Code