Report the capture window from feed clocks and pass channel and mode to the eval hook (#98) - #99
Merged
Merged
Conversation
…ode to the eval hook (#98) model-eol owns the deadline and the eval-hook contract, not migration verification. This adds the thin slice decided in issue 98. Findings gain an in-memory capture window computed from feed dates only: the applied clock's shutdown as the last day the old model answers, plus every other clock (publisher or distribution row) with a shutdown date after today and a status other than retired. Rows without a date are never listed. The human check line appends "[still answers via aws-bedrock until 2026-10-14]" to retiring and retired findings when such a clock exists. The check JSON document does not carry the field: the 0.1 report shape published by 0.5.2 is kept byte-compatible for default output, and adding a field there is a separate contract decision recorded in docs/DESIGN_EVAL_BOUNDARY.md. The bot adds a "## Capture window" section to PR and issue bodies from the same computation and passes MODEL_EOL_VIA (the plan's channel, empty for the publisher API) and MODEL_EOL_EVAL_MODE (evaluate) to the eval hook. The hook refuses any mode other than evaluate; capture is reserved and documented, not implemented. Implemented by Codex (gpt-5.6-luna) from a Claude brief. Claude removed the check-schema change Codex made after it broke the pinned 0.5.2 contract test, moved the capture tests to findingFromRef, wrote the design note, and tightened the docs. Full npm test green.
…erally Review pass 1 found three gaps in the capture window work. The capture section reads distribution rows that are not plan items, so a row changing status, date, or precision left an open PR or issue at skip-unchanged with a stale "still answers via" line. The group digest now folds in the entry's shutdown, precision, and every distribution row's via, shutdown, status, and precision. Existing bot PRs and issues will refresh once after this ships because their recorded digest no longer matches. markdownCode escaped underscores and other characters to HTML entities, and code spans show entities literally, so a channel or model ID such as custom_hub rendered as custom_hub. Code spans now drop backticks and line breaks and keep everything else as typed. A pre-existing test that asserted the corrupted form now asserts the literal one. The final capture bullet told the reader to capture "before that date" even when no clock in the feed still answered. It is now conditional. Claude implemented these directly from the reviewer's inputs; each is a regression test. Full npm test green.
…ated clocks Review pass 2 left two P2 residuals. Distribution rows joined the group digest in feed order, so a feed that only reordered rows between the evaluate and publish phases made the publish job refuse the eval results manifest on a digest mismatch; the rows now pass through the sorted item digest. The closing capture bullet said no clock still answered whenever the alternatives list was empty, but an active distribution without a date is unknown, not closed; the bullet now says the feed lists no dated clock. Both inputs are regression tests.
… unit Review pass 3 found that a date-independent digest let an issue or PR keep "still answers via aws-bedrock until <date>" after that date passed. The bot metadata now records capture_expires, the first date on which the capture section changes, and skip-unchanged requires that date to still be in the future. The digest stays date-independent, so the evaluate and publish phases keep binding the same eval results manifest. sortedJsonDigest ordered items with localeCompare, which returns zero for canonically equivalent Unicode strings, so reversing two such rows changed the digest. It now compares code units. The README and the design note said a floor was the last capture day; they now say a tentative or earliest floor is "at least until". Regression tests cover PR and issue expiry on the expiry date, the expiry metadata field, and locale-equal and case-different digest order.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #98.
model-eol owns the deadline and the eval-hook contract, not migration verification. This is the thin slice from the 2026-09-09 design review, recorded in
docs/DESIGN_EVAL_BOUNDARY.md.What changes
checkline appends[still answers via aws-bedrock until 2026-10-14]when such a clock exists. A tentative or earliest floor renders asat least until.check --jsondocument does not carry the field. Default output must keep validating against the pinned 0.5.2 schemas (test/fixture/schemas-0.5.2), and the fixture repo already has five findings with alternatives, so no "emit only when informative" rule keeps that pin green. Addingcaptureto the 0.1 report shape is a separate contract decision, noted in the design doc. SPEC permits additive fields; the PR 91 precedent kept default output byte-compatible. Your call, not mine.## Capture windowsection from the same computation. Distribution rows now join the group digest (order-independent) so a row change refreshes the body, and metadata recordscapture_expires, the first date on which the section changes, so an expired alternative refreshes the body even though the digest is date-independent. The evaluate and publish phases keep binding the same eval results manifest.MODEL_EOL_VIA(the plan's channel, empty for the publisher API) andMODEL_EOL_EVAL_MODE(evaluate;capturereserved, refused today). Documented in README,docs/DESIGN_BOT.md, and the bot-config schema.markdownCodeentity-escaped underscores, and code spans show entities literally, socustom_hubrendered ascustom_hub. It now drops backticks and line breaks and keeps everything else. Pre-existing; found by review.sortedJsonDigestusedlocaleCompare, which returns zero for canonically equivalent Unicode strings. It now compares code units.Operational note
Existing bot PRs and issues will refresh once after this ships, because their recorded
feed_digestno longer matches the new digest input.Verification
npm testgreen at every commit.claude-sonnet-4-20250514showsRETIRED 2026-06-15 ... [still answers via aws-bedrock until 2026-10-14]on the direct scan andRETIRES 2026-10-14 (35 days) [via aws-bedrock]under--via aws-bedrock, both from the feed.Implemented by Codex (gpt-5.6-luna) from a Claude brief; Claude reverted the check-schema change, moved the capture tests to
findingFromRef, wrote the design note, and closed the review findings.