Repository navigation
Let visitors choose a test's shape, rate, duration and request mix - #4
Merged
Merged
Conversation
- Add bench/k6/src/settings.ts: read and validate RATE, DURATION, SHAPE (steady, ramp, spike) and MIX (reads, writes, health), and build the arrival-rate scenario - Fit every shape exactly in its duration, including the spike's two 2s jumps - Size k6 virtual users from the rate and cap them (300 per server), so big tests fit the runner's memory limit - Turn the request mix into named mixes in workload.ts: mostly reads, write-heavy, and health checks only - Tag every metric with shape, mix and rate, and record shape and mix in run summaries - Document the settings in bench/README.md
- Add a settings menu to the single-test and head-to-head pages: shape, peak rate, duration and request mix, with a live request estimate - Accept only exact menu values (infra/runner/settings.go) and refuse tests over RUNNER_MAX_TOTAL_RATE across all servers, counted as invalid-settings rejections - Pass the settings to k6, the trace span, the logs and the test id (e.g. express-ramp-1000rps-web-<time>) - Poll for a free runner on both forms instead of reloading, so a visitor's choices survive - Replace RUNNER_SCRIPT and RUNNER_MATCH_RATE with RUNNER_RATES, RUNNER_DURATIONS and RUNNER_MAX_TOTAL_RATE (default 2000) - Cap the runner container, where k6 runs, at 768 MB BREAKING CHANGE: RUNNER_SCRIPT and RUNNER_MATCH_RATE are no longer read; visitors choose the shape and every server in a test gets the same rate. RUNNER_RATE and RUNNER_DURATION now only preselect a menu value.
- Add shape, mix and peak-rate columns to Run Comparison - Limit the leaderboard's fixed-rate p99 to steady, mostly-reads runs so it stays like for like - Rewrite the test-conditions note for visitor-chosen settings - Document the visitor menus, the total-rate budget and the runner memory cap in infra/README.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Every visitor test was 500 req/s for 30 s (15,000 requests), fixed in
.env. Visitors could not explore how a server behaves under more load, a ramp, a spike, or a different request mix.What visitors can pick now
On both the single-server page and Head to head, from fixed menus:
RUNNER_RATES)RUNNER_DURATIONS)The page shows a live estimate ("About 45,000 requests in total, 15,000 to each server") and greys out rates that would break the budget for the number of servers picked.
Validation and safety
invalid-settingsrejection. k6 checks the settings again (bench/k6/src/settings.ts).RUNNER_MAX_TOTAL_RATEreq/s across all its servers (default 2000). Measured locally: generating 3000 req/s took k6 about 1.9 CPU cores, which on the 2-core VPS would starve the servers and the other sites for the whole test.Dashboards
Every metric is tagged with
shape,mixandrate; Run Comparison shows them as columns; test ids read likeexpress-ramp-1000rps-web-<time>; the leaderboard's fixed-rate p99 only counts steady, mostly-reads runs.Tested locally (full stack)
rate=500;rm -rfinjection and an over-budget race are all refused.Breaking change
RUNNER_SCRIPTandRUNNER_MATCH_RATEare no longer read.RUNNER_RATEandRUNNER_DURATIONnow only preselect a menu value. The VPS'sinfra/.envkeeps working; the old keys are ignored.