test(promql-compliance): add olly-bench suite and p75 quantile queries - #739
Merged
Merged
Conversation
Add the olly-bench handwritten_v2 queries (#738) as a separate suite with a stand-in dataset. Durations are shrunk to fit the fixture; query structure is unchanged. Most queries are not yet planned, so the suite is excluded from CI: `make run-all` now runs every suite and the CI workflow uses the new `make run-for-ci`. Also add p75 quantile_over_time / quantile queries to the quantiles suite. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
suites/olly-bench.yaml+datasets/olly-bench.yaml: the 25 unique olly-benchhandwritten_v2queries tracked in Tracking: asap-planner query coverage for olly-bench #738. Query structure is unchanged; durations are shrunk to fit the fixture ([1h]/[6h]/[12h]/[24h]→[5m], offsets →5m,[6h:]→[10m:]). Duplicate lines 17 and 22 are omitted; names carry the original line numbers.make run-allruns every suite,make run-for-ciruns onlyCI_CASES. CI now usesrun-for-ci, since most olly-bench queries are expected to fail until Tracking: asap-planner query coverage for olly-bench #738 lands.quantile_over_time/quantile/quantile by (job)/quantile by (job, instance)queries to the quantiles suite.Notes
topk/sort_deschave unique answers, and jobworkerhas zero 5xx so> 0actually filters.sort_descordering is not verified: the comparator sorts series before comparing.Test plan
go test ./...inrunnerandseederquantilessuite: passes, including all four p75 queriesolly-benchsuite: 3/25 pass (lines 5, 6, 9), matching Tracking: asap-planner query coverage for olly-bench #738; the rest fail withNo result for queryfrom ASAPQuery. Prometheus reference parity passes on every query.Refs #738
🤖 Generated with Claude Code