Skip to content

[Deferred] test: validate accuracy evidence independently of candidate costs - #777

Draft
zzylol wants to merge 36 commits into
test/workload-statisticsfrom
test/accuracy-evidence
Draft

zzylol wants to merge 36 commits into
test/workload-statisticsfrom
test/accuracy-evidence

Conversation

@zzylol

@zzylol zzylol commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Deferred follow-up. Not required for the current #728 → #742 → #775 synthetic-selection and data-plane execution milestone. Online ERP feedback and runtime replanning are explicitly out of scope for that milestone.

Make accuracy-evidence admission independently reviewable from candidate pricing.

Before this PR: evidence applicability was mainly covered inside ERP implementation tests.

After this PR: public-adapter integration tests reject mismatched distribution, implementation, metric, parameters, insufficient trials and excessive error even for cheap candidates; distinguish empirical admission from theoretical/exact fallback; and retain admitted record identity.

Validation: three accuracy_evidence tests passed. Records are explicitly synthetic contract fixtures, not calibration results. Existing readout-specific, formal-guarantee and freshness checks remain in place.

Base: workload-statistics tests. Next: resource-measurement tests.

@zzylol zzylol changed the title test: validate accuracy evidence independently of candidate costs [Deferred] test: validate accuracy evidence independently of candidate costs Sep 28, 2026
@zzylol
zzylol marked this pull request as draft September 28, 2026 02:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant