Cut contributor tests that pin shape instead of a contract

[mzored]

Summary

This audit covers the contributor side: scripts/, tests/, evals/, docs/evidence.md and docs/decisions.md. Every deterministic assertion was sorted into an AGENTS.md "Checks" class: a contract (fails the run), lint (non-blocking), or shape (deleted). The yardstick was the 7.0.0 kernel Engineering standard and references/testing.md: name the production change that would make a test fail.

This PR changes only tests, scripts and docs. plugins/skiphow/ is untouched, so VERSION stays where it is (PATCH, same as #151 and #152).

	Before	After
Full python scripts/check.py	44.2 s, 426 tests	17.1 s, 345 tests
tests/*.py lines	7,397	6,331
scripts/*.py lines	4,610	4,606
Test + tooling lines per package line (package is 1,403 lines)	8.6×	7.8×

Delete

• tests/test_package.py, the whole file. Each test either re-ran a validator that check.py offline_checks() already runs on the real tree (plugin static, version, marketplaces, site, links, portability), or pinned shape: the dogfood symlink, release-not-in-README, marketing-description parity, and a hook test that returns early. The one unique check, that the skill-discovery matrix keeps both positive and negative cases, moved to test_evals_corpus.py.
• tests/test_host_pilot_capture.py. It tests the one-off driver of a finished receipt. The driver stays as part of the receipt record.
• evals/receipts/capture-synthetic-pilot.json. It is synthetic and nothing references it.
• tests/test_checks.py, 28 tests:
  • duplicates of the real-tree run;
  • argv and call-order pins (git diff --check triple, git ls-files flags, version-delta call list);
  • source-reading tests ("pip" appears once, no venv import);
  • tautologies (site copy not pinned to a phrase);
  • tests of the repo's own tooling (the bytecode probes that spawn pytest, the durations report, and the dependency fail-fast);
  • edge cases already covered by a sibling negative test.
• tests/test_checks.py, parameters. Manifest-component cases went from 15 to 3: the validator is an allowlist with two error paths, and these three cases cover both. Kernel-link cases went from 9 to 4, and accepted-link cases from 2 to 1.
• tests/test_hosts.py, 6 tests. They pinned argparse behavior, the matrix header sentence, and diff-message formatting.
• tests/test_workflows.py, 7 per-job pins. They are replaced by one general rule: no job holding any write permission checks out code, sets up Python, or runs install, python or scripts. The SHA pin, per-job permissions and read-only CI tests are kept. So are the exact-SHA guard on the manual site repair and the release guard that the tag matches VERSION and sits on main.
• tests/test_activation.py, 1 test. It checked the helper's file mode.
• tests/test_evals_corpus.py. Removed the roster assertions that pinned spec section numbers and the journey count of 3. Kept the check that some case can tell activation from its absence.

Demote to lint (scripts/check.py validate_site)

• Site metadata is now lint. That covers html lang, nonempty and unique titles and descriptions, JSON-LD content (@type, url, description, parse), and the repository link. Decision row 38 already says presentation is lint or unchecked, so these blocking checks contradicted a live decision.
• The site/.nojekyll requirement is gone. Both deploy paths use upload-pages-artifact/deploy-pages, so no Jekyll build runs. The file itself stays.
• Still errors: parse, canonical URL and og:url, no client runtime, local link resolution, the sitemap, the robots.txt rule, and the evidence-scope binding.

Keep (and the contract each protects)

• tests/test_contract.py security pins: the security boundary. Untouched.
• Corpus-validity tests: the semantic validity of the behavioral corpus. They stay, along with the preflight, grader and simplification-fixture tests, which are kept under decision row 37.
• Version-bump core and rename-out tests: the release invariant that plugins/skiphow/ cannot change without a VERSION bump.
• Marketplace identity, third-party/MIT provenance, and the openai metadata policy: package and host contracts.
• Hook schema tests, hostile-URI, FIFO, symlink-root, Windows paths, duplicate YAML/JSON keys, decoded-representation scans, and personal-path routes: security and portability hardening. They stay with their code pending the owner decisions below.

Docs

• decisions.md row 38 cited a "Spec 11" that no longer exists. It now cites the AGENTS.md assertion classes and records the demotion.
• evidence.md said "the absence of an executable package hook". It now says the safety shape of any shipped hook, which is what the check does.
• CONTRIBUTING.md: the sample focused-test target pointed at the deleted file. It now points at tests/test_contract.py.

Owner decisions (deferred, not done)

1. Should claim_eligibility.py, grade_catalog.py and their tests (about 2,000 lines) be frozen as the 4.1/4.2 ledger, or kept as a script with about 10 tests?
2. Should the capture_eval prepare/capture/main lifecycle be retired?
3. Should the check_hosts.py install/smoke path become a slim install on the release path, or should the static UNVERIFIED rows be accepted? Its remaining tests are tied to this answer.
4. Should these be kept as gates, demoted to lint, or deleted? Each is code plus its tests:
   • the dormant receipt validators (host receipt, committed identity);
   • the representation-decoding evasion layers in check.py (about 350 lines);
   • the hook validator, for a hook that does not ship;
   • the MIT text pin and the SOURCES schema roster;
   • the exact marketplace pins;
   • the evidence-page scope sentence;
   • runtime_portability_scan.
5. Should the per-release paragraphs in evals/README.md (about 330 lines, some with unique operator instructions) and the spec_refs data in cases.json, which point at a missing spec, be cleaned up?
6. Two corpus cases need a ruling:
   • cto-small-known-bug requires a new focused regression check, while the 7.0.0 case bug-covered-by-existing-check-no-new-guard and the kernel rule say not to add one. Which wins?
   • stewardship-read-only-operational-cost still cites the removed operations.md. Should it be updated or dropped?
7. Can the receipt directories that nothing cites be removed? One example is host-validation-420-20260906.

Verification

• Full python scripts/check.py passed on this candidate: 345 passed, 17.1 s.
• tests/test_workflows.py was rerun alone after its last edit, and nothing else changed. 6 passed.
• git diff --check is clean.
• No model was started. check_hosts.py was not run because package layout, manifests and installation are unchanged. Host rows stay UNVERIFIED.

🤖 Generated with Claude Code