Repository navigation
fix(bench): refuse to run when the disk is too full for Moon to take writes - #49
Merged
Merged
Conversation
…writes Moon's disk monitor stops taking writes when its volume has less than 5% free. laya-codex fails open on a refused score-cache write, so on a full disk every follow-up prompt is re-scored by the model instead of served from its session's cache. A run then times a hook that users, whose cache works, never pay. Benchmark v15 ran on a disk with 2% free. Moon engaged its write stall about 4 s after each arm's Moon started, just after the warm-up query, and none of the release candidate's 60 follow-ups was served from the cache (its httpx store holds 8 cache entries, written before the stall); v0.4.0's follow-up hooks took as long as its first prompts (0.54 s). Ranking and the injected code were unaffected, since the index was written before the stall, but the hook time per session counts both prompts at full cost (release candidate +0.14 s, v0.4.0 +0.53 s). run_bench now checks the volume of the output directory, where each arm's home and Moon data live, and the default laya-codex home for an arm on the default binary, and refuses to start when either has less than 6% free: Moon's floor plus one point, so a run does not cross it midway. author: Tin Dang
author: Tin Dang
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Benchmark v15 ran with 2% of the disk free. Moon's disk monitor stops taking writes under 5%, and laya-codex fails open on a refused score-cache write. So every follow-up prompt in both laya-codex arms was re-scored by the model instead of served from its session's cache:
That timed a hook that users, whose cache works, never pay: about +0.14 s per session for the release candidate and +0.53 s for v0.4.0. Ranking and the injected code were unaffected, because the index was written before the stall.
What
run_benchchecks the volume of the output directory, where each arm's home and Moon data live, and of the default laya-codex home for an arm on the default binary. It refuses to start when either has less than 6% free: Moon's floor plus one point, so a run doesn't cross it midway. A path that doesn't exist yet is checked on its nearest existing parent.Lever: honest measurement.
Tests
bench/test_bench.pyDiskRoomForMoon: the floor is reported with its free share, the margin is kept, and missing paths are checked on their parent.disk_shortreports the shortfall.