Skip to content

Test coverage for manifest/baseline and CI hardening - #17

Open
patsypppe wants to merge 2 commits into
mainfrom
chore/improvements-2026-10
Open

patsypppe wants to merge 2 commits into
mainfrom
chore/improvements-2026-10

Conversation

@patsypppe

Copy link
Copy Markdown
Owner

Summary

  • Unit tests for manifest/build.py (40% -> covered) and gate/baseline.py (0%), using a throwaway git repo; no Docker or network.
  • ci.yml: workflow-level permissions: contents: read, job timeout-minutes.
  • Add .github/dependabot.yml (github-actions, uv; weekly).

Test plan

  • make check green (325 unit tests, was 311)

🤖 Generated with Claude Code

patsypppe and others added 2 commits October 3, 2026 20:56
Adds unit tests for manifest/build.py (was 40%) and gate/baseline.py (was 0%).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 4, 2026

Copy link
Copy Markdown

Meridian gate

✅ PASS — suite pass^k moved -0.000 (0.486 → 0.486), within the 0.030 tolerance

Failing trials in this run
  • expired-coupon — SELECT status FROM orders WHERE id = 1042 returned [['pending']], expected [['invoiced']]
  • expired-on-bulk-order — SELECT status FROM orders WHERE id = 1045 returned [['pending']], expected [['invoiced']]
  • expiry-boundary — SELECT status FROM orders WHERE id = 1046 returned [['pending']], expected [['invoiced']]
  • happy-path — SELECT status FROM orders WHERE id = 1041 returned [['pending']], expected [['invoiced']]
  • stale-coupon — SELECT status FROM orders WHERE id = 1044 returned [['pending']], expected [['invoiced']]
  • unknown-coupon — SELECT status FROM orders WHERE id = 1047 returned [['pending']], expected [['invoiced']]
task baseline pass^3 head pass^3
expired-coupon 0.400 0.400
expired-on-bulk-order 0.400 0.400
expiry-boundary 0.400 0.400
happy-path 0.400 0.400
missing-field 1.000 1.000
stale-coupon 0.400 0.400
unknown-coupon 0.400 0.400
baseline head
suite pass^3 0.486 0.486

Sensitivity. Not estimated: all 7 tasks moved by the same amount, so there is no spread to estimate sensitivity from. Treat the verdict as directional.

n=5 · k=3 · status complete · harness errors 0.0% · cost 51c

head run run-1791075481-e21cb1 · baseline run run-1791075441-d2e0c2 · manifest sha256:c3a56945113a5ef7… · reproduce with meridian replay run-1791075481-e21cb1

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant