新增完整交付协议与真实 Agent 验收评测 - #97
Merged
Merged
Conversation
Owner
Author
新上下文独立验收 PASS精确head 2728a1f,base 0961c8e。A1-A4全部PASS:完整真实Git交付及明确模拟tracker;7协议场景、8额外恢复/清理组;3个已归档真实Agent会话和成功实现经新oracle重放;Node22/26各258/258,全部本地CI门。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
结果
新增可执行交付评测:真实Git分支、bare remote和worktree配合显式模拟Issue/PR tracker,覆盖正常交付及验收漂移、兄弟主干变化、真实push后命令报错、提前关闭、脏工作树和真实进程中断恢复。验收绑定base/head/contract,关闭前核对main与精确资源清理。
独立的真实模型模式运行编码Agent和新上下文只读验收Agent;要求轨迹有实际测试命令退出码,不能仅凭最终回答PASS。模型前置缺失输出BLOCKED,输出目录拒绝复用,产物脱敏并保存哈希。该模式后续交付由本地适配器编排,不声称真实GitHub全链由模型自主完成。
验证
/Users/carver/workspace/mindcarver/test/thinloop-delivery-eval/20260905-issue92/,不会随工作树清理消失。页面不适用。独立最终验收、远端CI、合并和清理由父Execute负责;本PR不自动关闭Issue。单次fixture不用于推断整体有效性、自然路由能力或Token优势。
首轮独立验收修复
补齐完整index/工作区/已有提交及最终diff scope门,保护原始测试并运行内置独立行为断言;只认独立测试进程、完整TAP结果和匹配退出码,拒绝打印命令/掩盖失败。直接回归覆盖首轮两个P2反例及隐藏index变化。子测试环境清除NODE_TEST_CONTEXT,防止递归检测跳过实际测试。Node22与Node26均258/258 PASS,原真实模型三条轨迹通过更严格门,没有新增付费模型请求;等待新head独立复验。
独立行为判定完整性
直接行为检查在动态导入候选前捕获输出函数并生成私有nonce,使用原始值比较而非共享assert方法;父进程核验完整start/result与退出码。回归证明覆盖assert和提前exit(0)均不能接受,原正确模型归档仍通过新检查。修复针对既有oracle完整性,不扩展为通用恶意代码沙箱。当前Node22/Node26均258/258 PASS,等待新head fresh复验。
Refs #92