fix(ci): Cap check-container cores and raise the timeout floor - #2875
Merged
Conversation
Two remedies from the contention analysis of run 32574134229, where all seven timeouts sat at the 1200 s floor on 4-core shards running at load 5-13. Check containers now run with _R_CHECK_LIMIT_CORES_=TRUE, the value CRAN's own check machines use: a cluster or fork call asking for more than 2 workers errors instead of spawning them. Without it a test suite sizing itself from parallel::detectCores() sees the runner's 4 cores and fans out 4 workers per check, and four concurrent checks amplified each other into exactly the hot windows the timeouts died in. Both halves get the identical setting, so the old-vs-new comparison is unaffected. The timeout floor rises from 20 to 30 minutes: a saturated shard runs each check at roughly half speed (measured queue-vs-pair inflation median 1.5x, p90 1.8x, up to ~3x in the hottest windows), and the old floor killed four packages that finish in 7-12 minutes uncontended. The cost is bounded -- a genuinely hung check holds a worker ten extra minutes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UB7YutLzVWU7xCTYUvF3kF
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Prepared with Claude Code (LLM-assisted).
Two remedies from the contention analysis of the revdepx validation run 32574134229, where all seven timeouts sat exactly at the 1200 s floor while the 4-core shards ran at 1-minute loads of 5–13.
_R_CHECK_LIMIT_CORES_=TRUEin every check container (check-half.sh, overridable through the environment). This is the value CRAN's own check machines use: a cluster or fork call asking for more than 2 workers errors instead of spawning them. Without it, a test suite that sizes itself fromparallel::detectCores()sees the runner's 4 cores and fans out 4 workers per check — and with 4 concurrent checks per shard, that fan-out is what drove the load to 5–13 and produced the hot windows every timeout died in (the timed-out packages' windows ran at median load 7.5–11.4). Both halves of every comparison get the identical setting, so old-vs-new verdicts are unaffected; checks simply behave the way CRAN runs them.Timeout floor 20 → 30 minutes (
REVDEPX_TIMEOUT_MIN_MINUTESdefault). A saturated shard runs each check at roughly half speed — the measured inflation of queue-engine half-times over the pair engine's wall clock for the same 1,923 packages is 1.5× at the median, 1.8× at p90, and up to ~3× in the hottest windows. The old 20-minute floor killed four packages (orbital, PortfolioTesteR, SMMAL, xpect) that finish in 7–12 minutes uncontended. The cost is bounded: a genuinely hung check holds one worker ten extra minutes.Both changes live in the shared
revdepxcore plus therevdep4.yamldefaults, so they apply to the merged queue engine now and to the pair engine (#2856) as-is on its merge — onlyrevdep3.yaml's own|| '20'floor default would want the same one-line bump on that branch.🤖 Generated with Claude Code
https://claude.ai/code/session_01UB7YutLzVWU7xCTYUvF3kF
Generated by Claude Code