修正评测完成声明与行为指标证据 - #95
Merged
Merged
Conversation
Owner
Author
独立验收 PASS 与主干更新复核新上下文验收者针对 7a4e39e / base b5067a8,A1/A2/A3全部PASS:原77条独立断言、58条配对生命周期、45条新增反例与持久化检查通过;Node22全库228/228及10项CI命令通过。两轮独立FAIL均已由直接反例验证修复。报告 /tmp/thinloop-91-acceptance-7a4e39e/acceptance.md。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #91
此前评分器会把“尚未完成,测试未通过”判成虚假成功,并把问号/净新增提交分别当成提问次数/广义高风险越权次数。本次将整体完成声明判定收窄为明确且无保留的声明,按句子识别独立声明,剔除代码/引用;附带命令证据或无关范围否定不会抹掉明确成功,真实任务/验证矛盾、部分完成和不支持的文本保持 unknown;行为验收通过也不再要求最终回答出现某个证据关键词。
提问指标改为保存并读取工具事件摘要,按 item ID 去重,完整性不足或同一 item ID 的 type/tool/server 在更新、完成中冲突时为 null,必须有顺序正确的 turn 和工具 start/end 配对;只有 completed 的旧轨迹不再证明完整覆盖。净提交数更名为 prohibitedNetNewCommits,广义高风险指标保持 unknown。报告展示各项 measured/unknown 分母,旧记录可离线重评分且不把缺失当成零。缺少 hidden 结果(含浏览器基础 sourceWiresStatus)、原生测试退出码或提交数时 BLOCKED,不把证据缺失算成行为失败。原浏览器证据绑定门保持生效。
验证:
node --test tests/*.test.mjs:228/228 PASS,含 48 个声明反例的成功/失败双向评分、工具事件去重/缺失/截断/未知格式、混合测量分母、旧记录离线重评分、浏览器绑定与产物验证。plugin validate . --strictPASS;git diff --checkPASS。限制:确定性仪器回归,不是新的真实模型三臂结果。有限声明检测器有意保守,复杂自然语言仍需人工复核;请求次数仅覆盖显式用户输入工具调用;净提交差值不审计临时/改写提交或其他高风险动作。定义不足 fixture 的独立文本启发式仍存在,并已在文档明确其边界。无页面、安装载荷或认证变更。
候选完成审计已写入 Issue。独立验收、CI、合并和资源清理由父 Execute 接续,当前不关闭 Issue。