From 8a226ea319da93ee247ac2847d5648e7dce0af83 Mon Sep 17 00:00:00 2001 From: mchwang Date: Wed, 23 Sep 2026 17:55:45 -0700 Subject: [PATCH 01/15] Freeze paired review experiment inputs --- docs/experiments/manual-assignment.md | 19 +++++++++++++++++++ docs/experiments/review-protocol.md | 20 ++++++++++++-------- 2 files changed, 31 insertions(+), 8 deletions(-) create mode 100644 docs/experiments/manual-assignment.md diff --git a/docs/experiments/manual-assignment.md b/docs/experiments/manual-assignment.md new file mode 100644 index 00000000..71956056 --- /dev/null +++ b/docs/experiments/manual-assignment.md @@ -0,0 +1,19 @@ +# Manual real-issue assignment + +This prerequisite exercise uses real issue [#10](https://github.com/codeabovelab/codeboost/issues/10) and merged [PR #17](https://github.com/codeabovelab/codeboost/pull/17). It is an attribution exercise, not a timed trial and not evidence for the go/no-go thresholds. + +- Base: `5a2685cc709ca8f9f384e35e4d733012c7290d69` +- Head: `442147d1bc9a33edeca1a605c9de3ada13f397f1` +- Ledger setup: the squash commit was intentionally recorded as foreign so every resulting segment required an explicit manual assignment. +- Plan: P1 rename-reassignment regression, P2 literal-path regression, P3 evidence record, and P4 durable review rules. + +| Plan item | Declared file | Assigned segments | Final attribution | Final scope | +|---|---|---:|---|---| +| P1 | `test/review.test.ts` | 4 text | Attributed | In scope | +| P2 | `test/plant.test.ts` | 4 text | Attributed | In scope | +| P3 | `docs/experiments/review-summary-edge-cases.md` | 1 file + 1 text | Attributed | In scope | +| P4 | `AGENTS.md` | 2 text | Attributed | In scope | + +Codeboost exposed 12 segments before assignment, all in the Unplanned row. Each segment mapped to exactly one item through that item's declared file contract. After the assignments, the per-item segment counts were 4, 4, 2, and 2; all four rows reported `✓ Attributed` and `✓ In scope`. No segment was unclear or multiply owned, so this exercise produced no ambiguity fixture. The earlier rename and literal-path concerns already have focused fixtures in `test/review.test.ts` and `test/plant.test.ts`. + +The assignment was rerun from a detached worktree at the exact head using a fresh SQLite review store. The stored choices, rather than a hand-written diff classification alone, produced the final attribution and scope results above. diff --git a/docs/experiments/review-protocol.md b/docs/experiments/review-protocol.md index 8291265e..951cdf9f 100644 --- a/docs/experiments/review-protocol.md +++ b/docs/experiments/review-protocol.md @@ -1,19 +1,23 @@ # Plan-indexed review: go/no-go protocol -Status: **not started; issue pairs and human reviewer still required**. This document does not claim a passed gate. Complete the pair table and commit its exact revision before any timed review. No pair or threshold may change once the first review begins. +Status: **pair selection frozen; implementations and reviews not started**. Max Hwang (`mchwang`) is the blinded reviewer and Codex is the experiment operator. This document does not claim a passed gate. Complete each row's PR and SHA fields and commit that exact revision before the first timed review. No issue, assignment, order, or threshold may change once the first review begins. ## Pair selection and order Choose four pairs of comparable, small real issues in a repository owned by the reviewer. Match each pair on scope, language, and estimated review complexity before implementing either issue. Record issue URLs, PR URLs, base/head SHAs, and which review method each gets. Do not reuse the demo fixture as an experimental issue. -| Pair | Real issue A / PR | Real issue B / PR | A method | B method | +All eight issues are in `codeabovelab/guardyx-gstack`. A/B placement within each pair was randomized once before implementation; the method pattern remains the alternating pattern approved in the protocol. + +| Pair | Real issue A / PR / base..head | Real issue B / PR / base..head | A method | B method | |---|---|---|---|---| -| 1 | Pending selection | Pending selection | Raw GitHub diff | codeboost | -| 2 | Pending selection | Pending selection | codeboost | Raw GitHub diff | -| 3 | Pending selection | Pending selection | Raw GitHub diff | codeboost | -| 4 | Pending selection | Pending selection | codeboost | Raw GitHub diff | +| 1 — MCP discovery | [#556](https://github.com/codeabovelab/guardyx-gstack/issues/556) / PR pending / SHA pending | [#554](https://github.com/codeabovelab/guardyx-gstack/issues/554) / PR pending / SHA pending | Raw GitHub diff | codeboost | +| 2 — companion documentation | [#493](https://github.com/codeabovelab/guardyx-gstack/issues/493) / PR pending / SHA pending | [#176](https://github.com/codeabovelab/guardyx-gstack/issues/176) / PR pending / SHA pending | codeboost | Raw GitHub diff | +| 3 — agent-key security | [#506](https://github.com/codeabovelab/guardyx-gstack/issues/506) / PR pending / SHA pending | [#443](https://github.com/codeabovelab/guardyx-gstack/issues/443) / PR pending / SHA pending | Raw GitHub diff | codeboost | +| 4 — compliance export | [#504](https://github.com/codeabovelab/guardyx-gstack/issues/504) / PR pending / SHA pending | [#503](https://github.com/codeabovelab/guardyx-gstack/issues/503) / PR pending / SHA pending | codeboost | Raw GitHub diff | + +The pairs are matched before implementation on repository, primary implementation language, subsystem, and expected review surface. Pair 1 changes MCP discovery behavior; pair 2 changes operational/compliance documentation without runtime behavior; pair 3 changes agent-key lifecycle across the API and UI; pair 4 adds per-invocation compliance evidence across export and PDF surfaces. If implementation discovery makes a pair materially unequal or blocked, stop before the first timed review and record a new frozen protocol revision rather than silently substituting an issue. -The reviewer must not implement the paired changes or inspect the plants before deciding on the PR. An operator prepares plans and commits by hand, records trusted ledger ownership, and supplies both the plan and code to each review method. The earlier one-issue manual assignment also remains required: record how every change maps to the attribution table and turn unclear cases into fixtures. +The reviewer must not implement the paired changes or inspect the plants before deciding on the PR. The operator must not expose plant inputs, `sealed.json`, mappings, planted locations, or unblinded implementation notes in the PR, review package, task messages, or protocol updates. An operator prepares plans and commits by hand, records trusted ledger ownership, and supplies both the plan and code to each review method. The earlier one-issue manual assignment is recorded in `manual-assignment.md`; all 12 segments were assigned and in scope, with no unclear case requiring another fixture. ## Planting @@ -37,4 +41,4 @@ Report declared-file catch rate for both methods without a pass threshold. Stop ## Current evidence -Engineering tests prove browser interactions, persistence, attribution, and planting mechanics. They do **not** establish human review speed or catch rates. No real issue pairs, timed decisions, or catch-rate results exist yet. +Engineering tests prove browser interactions, persistence, attribution, and planting mechanics. The separate manual assignment proves that one real merged issue can be completely classified through stored review choices. Neither establishes human review speed or catch rates. Four Guardyx issue pairs are selected, but no paired implementation, timed decision, or catch-rate result exists yet. From a36162c87184a1272dc3f64df7042f682b998321 Mon Sep 17 00:00:00 2001 From: mchwang Date: Wed, 23 Sep 2026 17:57:45 -0700 Subject: [PATCH 02/15] Replace telemetry-blocked experiment pair --- docs/experiments/review-protocol.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/experiments/review-protocol.md b/docs/experiments/review-protocol.md index 951cdf9f..33f1082d 100644 --- a/docs/experiments/review-protocol.md +++ b/docs/experiments/review-protocol.md @@ -10,12 +10,14 @@ All eight issues are in `codeabovelab/guardyx-gstack`. A/B placement within each | Pair | Real issue A / PR / base..head | Real issue B / PR / base..head | A method | B method | |---|---|---|---|---| -| 1 — MCP discovery | [#556](https://github.com/codeabovelab/guardyx-gstack/issues/556) / PR pending / SHA pending | [#554](https://github.com/codeabovelab/guardyx-gstack/issues/554) / PR pending / SHA pending | Raw GitHub diff | codeboost | +| 1 — small backend reliability/reporting | [#335](https://github.com/codeabovelab/guardyx-gstack/issues/335) / PR pending / SHA pending | [#172](https://github.com/codeabovelab/guardyx-gstack/issues/172) / PR pending / SHA pending | Raw GitHub diff | codeboost | | 2 — companion documentation | [#493](https://github.com/codeabovelab/guardyx-gstack/issues/493) / PR pending / SHA pending | [#176](https://github.com/codeabovelab/guardyx-gstack/issues/176) / PR pending / SHA pending | codeboost | Raw GitHub diff | | 3 — agent-key security | [#506](https://github.com/codeabovelab/guardyx-gstack/issues/506) / PR pending / SHA pending | [#443](https://github.com/codeabovelab/guardyx-gstack/issues/443) / PR pending / SHA pending | Raw GitHub diff | codeboost | | 4 — compliance export | [#504](https://github.com/codeabovelab/guardyx-gstack/issues/504) / PR pending / SHA pending | [#503](https://github.com/codeabovelab/guardyx-gstack/issues/503) / PR pending / SHA pending | codeboost | Raw GitHub diff | -The pairs are matched before implementation on repository, primary implementation language, subsystem, and expected review surface. Pair 1 changes MCP discovery behavior; pair 2 changes operational/compliance documentation without runtime behavior; pair 3 changes agent-key lifecycle across the API and UI; pair 4 adds per-invocation compliance evidence across export and PDF surfaces. If implementation discovery makes a pair materially unequal or blocked, stop before the first timed review and record a new frozen protocol revision rather than silently substituting an issue. +The pairs are matched before implementation on repository, primary implementation language, scope, and expected review surface. Pair 1 makes small Python backend changes with focused tests; pair 2 changes operational/compliance documentation without runtime behavior; pair 3 changes agent-key lifecycle across the API and UI; pair 4 adds per-invocation compliance evidence across export and PDF surfaces. If implementation discovery makes a pair materially unequal or blocked, stop before the first timed review and record a new frozen protocol revision rather than silently substituting an issue. + +Selection history: the first frozen draft paired #554 with #556. Before either implementation began, their required baseline check found that both depend on live 1,400-action/session telemetry unavailable from the operator's environment. They were replaced with #335 and #172, whose evidence can be reproduced locally. The replacement's A/B placement was randomized once; no timed review had started and no planted change existed. The reviewer must not implement the paired changes or inspect the plants before deciding on the PR. The operator must not expose plant inputs, `sealed.json`, mappings, planted locations, or unblinded implementation notes in the PR, review package, task messages, or protocol updates. An operator prepares plans and commits by hand, records trusted ledger ownership, and supplies both the plan and code to each review method. The earlier one-issue manual assignment is recorded in `manual-assignment.md`; all 12 segments were assigned and in scope, with no unclear case requiring another fixture. From d7f9b115b5dcd2192e54dec266cdf1eeb20ad292 Mon Sep 17 00:00:00 2001 From: mchwang Date: Wed, 23 Sep 2026 18:00:56 -0700 Subject: [PATCH 03/15] Verify experiment issues against current main --- docs/experiments/review-protocol.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/experiments/review-protocol.md b/docs/experiments/review-protocol.md index 33f1082d..63a3dbf4 100644 --- a/docs/experiments/review-protocol.md +++ b/docs/experiments/review-protocol.md @@ -10,14 +10,14 @@ All eight issues are in `codeabovelab/guardyx-gstack`. A/B placement within each | Pair | Real issue A / PR / base..head | Real issue B / PR / base..head | A method | B method | |---|---|---|---|---| -| 1 — small backend reliability/reporting | [#335](https://github.com/codeabovelab/guardyx-gstack/issues/335) / PR pending / SHA pending | [#172](https://github.com/codeabovelab/guardyx-gstack/issues/172) / PR pending / SHA pending | Raw GitHub diff | codeboost | -| 2 — companion documentation | [#493](https://github.com/codeabovelab/guardyx-gstack/issues/493) / PR pending / SHA pending | [#176](https://github.com/codeabovelab/guardyx-gstack/issues/176) / PR pending / SHA pending | codeboost | Raw GitHub diff | -| 3 — agent-key security | [#506](https://github.com/codeabovelab/guardyx-gstack/issues/506) / PR pending / SHA pending | [#443](https://github.com/codeabovelab/guardyx-gstack/issues/443) / PR pending / SHA pending | Raw GitHub diff | codeboost | -| 4 — compliance export | [#504](https://github.com/codeabovelab/guardyx-gstack/issues/504) / PR pending / SHA pending | [#503](https://github.com/codeabovelab/guardyx-gstack/issues/503) / PR pending / SHA pending | codeboost | Raw GitHub diff | +| 1 — companion documentation | [#493](https://github.com/codeabovelab/guardyx-gstack/issues/493) / PR pending / SHA pending | [#176](https://github.com/codeabovelab/guardyx-gstack/issues/176) / PR pending / SHA pending | Raw GitHub diff | codeboost | +| 2 — agent-key security | [#506](https://github.com/codeabovelab/guardyx-gstack/issues/506) / PR pending / SHA pending | [#443](https://github.com/codeabovelab/guardyx-gstack/issues/443) / PR pending / SHA pending | codeboost | Raw GitHub diff | +| 3 — asynchronous notifications | [#13](https://github.com/codeabovelab/guardyx-gstack/issues/13) / PR pending / SHA pending | [#429](https://github.com/codeabovelab/guardyx-gstack/issues/429) / PR pending / SHA pending | Raw GitHub diff | codeboost | +| 4 — compliance evidence | [#504](https://github.com/codeabovelab/guardyx-gstack/issues/504) / PR pending / SHA pending | [#507](https://github.com/codeabovelab/guardyx-gstack/issues/507) / PR pending / SHA pending | codeboost | Raw GitHub diff | -The pairs are matched before implementation on repository, primary implementation language, scope, and expected review surface. Pair 1 makes small Python backend changes with focused tests; pair 2 changes operational/compliance documentation without runtime behavior; pair 3 changes agent-key lifecycle across the API and UI; pair 4 adds per-invocation compliance evidence across export and PDF surfaces. If implementation discovery makes a pair materially unequal or blocked, stop before the first timed review and record a new frozen protocol revision rather than silently substituting an issue. +The pairs are matched before implementation on repository, primary implementation language, scope, and expected review surface. Pair 1 changes operational/compliance documentation without runtime behavior; pair 2 changes agent-key lifecycle across the API and UI; pair 3 adds medium asynchronous notification delivery across backend jobs and consumer-facing interfaces; pair 4 adds compliance evidence across stored invocation data, exports, mapping data, and PDF surfaces. If implementation discovery makes a pair materially unequal or blocked, stop before the first timed review and record a new frozen protocol revision rather than silently substituting an issue. -Selection history: the first frozen draft paired #554 with #556. Before either implementation began, their required baseline check found that both depend on live 1,400-action/session telemetry unavailable from the operator's environment. They were replaced with #335 and #172, whose evidence can be reproduced locally. The replacement's A/B placement was randomized once; no timed review had started and no planted change existed. +Selection history: the first frozen draft paired #554 with #556. Before either implementation began, their required baseline check found that both depend on live 1,400-action/session telemetry unavailable from the operator's environment. A proposed replacement included #335, but inspect-before-build then proved that `main` already contains its requested in-flight guard even though the issue remains open. A current-tree audit also excluded other stale open issues whose requested work has shipped. The table above contains only work confirmed absent from current `main`; every A/B placement was randomized after that audit. No timed review, experimental implementation, or plant had started before this final selection. The reviewer must not implement the paired changes or inspect the plants before deciding on the PR. The operator must not expose plant inputs, `sealed.json`, mappings, planted locations, or unblinded implementation notes in the PR, review package, task messages, or protocol updates. An operator prepares plans and commits by hand, records trusted ledger ownership, and supplies both the plan and code to each review method. The earlier one-issue manual assignment is recorded in `manual-assignment.md`; all 12 segments were assigned and in scope, with no unclear case requiring another fixture. From 576b973107bcdc6cd670e5bd5fd4667556535b32 Mon Sep 17 00:00:00 2001 From: mchwang Date: Wed, 23 Sep 2026 18:03:53 -0700 Subject: [PATCH 04/15] Make manual assignment evidence reproducible --- AGENTS.md | 1 + docs/experiments/manual-assignment.json | 18 ++++++++++++++++++ docs/experiments/manual-assignment.md | 2 +- docs/experiments/review-summary-edge-cases.md | 2 +- docs/implementation/read-only-review.md | 2 +- 5 files changed, 22 insertions(+), 3 deletions(-) create mode 100644 docs/experiments/manual-assignment.json diff --git a/AGENTS.md b/AGENTS.md index 67e77039..041b28ef 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -47,5 +47,6 @@ Every reproduced race requires a failing-before and passing-after regression. As - In evidence records, label cited commits as baselines, intermediate checkpoints, or validated heads. Keep final exact-head results in a place that can name the resulting commit, such as the PR body or CI record. - Reproduce summary-only review concerns or turn them into a concrete follow-up issue. Do not repeatedly patch vague wording without a failure case. - A validation fixture for a summary-only concern must assert the disputed intermediate representation or state before using a downstream outcome as evidence that the concern was exercised. +- Manual assignment evidence must include a reproducible per-segment manifest with stable choice ID, path, operation, assigned item, and final scope; aggregate counts alone are insufficient. - For each review round, record what changed, what was declined and why, and the regression evidence. Re-request review until a round returns no new findings. - Treat review-lesson extraction as a merge gate. Before invoking merge, classify every review finding in the PR body as: covered by an existing rule (cite it), captured by a new rule in this branch (cite it), or one-off (record why). Do not merge until this audit is complete and every required `AGENTS.md` update is included in the reviewed head. Omit rules that merely repeat existing guidance. diff --git a/docs/experiments/manual-assignment.json b/docs/experiments/manual-assignment.json new file mode 100644 index 00000000..617d31ab --- /dev/null +++ b/docs/experiments/manual-assignment.json @@ -0,0 +1,18 @@ +{ + "base": "5a2685cc709ca8f9f384e35e4d733012c7290d69", + "head": "442147d1bc9a33edeca1a605c9de3ada13f397f1", + "segments": [ + { "choice_id": "96c76cee6518b177f272d8dba1f8db83314d0c09b62f036c6d6f41de40227d5c", "kind": "text", "operation": "+", "path": "AGENTS.md", "before_row": "Unplanned", "item": "P4", "scope": "in-scope" }, + { "choice_id": "6000e022cb5f2c7fb602ba9dbd6b590cdf0672b1bdc984c036bf563423a86afd", "kind": "text", "operation": "+", "path": "AGENTS.md", "before_row": "Unplanned", "item": "P4", "scope": "in-scope" }, + { "choice_id": "aa5dbe642c11d5f5f6f1630c0f979006b8e0e3182da3e0aeee8485fa997784d1", "kind": "file", "operation": null, "path": "docs/experiments/review-summary-edge-cases.md", "before_row": "Unplanned", "item": "P3", "scope": "in-scope" }, + { "choice_id": "dfcf7097d83b6d55df0e4959f111370a96976b1c25f327e9d51838fc8224ceaf", "kind": "text", "operation": "+", "path": "docs/experiments/review-summary-edge-cases.md", "before_row": "Unplanned", "item": "P3", "scope": "in-scope" }, + { "choice_id": "7a5227f9a6be94d6c16d1ea058ae3a5d956a5b53f93333a15d3b778222a90218", "kind": "text", "operation": "-", "path": "test/plant.test.ts", "before_row": "Unplanned", "item": "P2", "scope": "in-scope" }, + { "choice_id": "142cf07bee22dbe7cf281b0d97efa49f9a502fed62701de8ec0c94500a94c788", "kind": "text", "operation": "+", "path": "test/plant.test.ts", "before_row": "Unplanned", "item": "P2", "scope": "in-scope" }, + { "choice_id": "36cb67aa87587c9159f19ce0bac78a4d67f1348f55108f99e1fed99e5273a414", "kind": "text", "operation": "+", "path": "test/plant.test.ts", "before_row": "Unplanned", "item": "P2", "scope": "in-scope" }, + { "choice_id": "88fada2e5b9a307f6db4fa6ccdbbf9e5bfbf00f26cbcacb33ac5bc141d74b85e", "kind": "text", "operation": "+", "path": "test/plant.test.ts", "before_row": "Unplanned", "item": "P2", "scope": "in-scope" }, + { "choice_id": "9f010f7ae66fd80d72c5d78d45d4effbcdac76ebb6dfadc4efd4e66ac4af9f86", "kind": "text", "operation": "-", "path": "test/review.test.ts", "before_row": "Unplanned", "item": "P1", "scope": "in-scope" }, + { "choice_id": "987cbec380909ee199bbe4df04a7d700c6f2a1f284c52f08120c159f8e92c35d", "kind": "text", "operation": "+", "path": "test/review.test.ts", "before_row": "Unplanned", "item": "P1", "scope": "in-scope" }, + { "choice_id": "6b276597aa7294f44e5a549e38d423876266fe6a9a502bb5be089db2223a8976", "kind": "text", "operation": "+", "path": "test/review.test.ts", "before_row": "Unplanned", "item": "P1", "scope": "in-scope" }, + { "choice_id": "c9379939cb69513a37cbb385440217283975294a2f8b340839d5f9d09ea126df", "kind": "text", "operation": "+", "path": "test/review.test.ts", "before_row": "Unplanned", "item": "P1", "scope": "in-scope" } + ] +} diff --git a/docs/experiments/manual-assignment.md b/docs/experiments/manual-assignment.md index 71956056..00029627 100644 --- a/docs/experiments/manual-assignment.md +++ b/docs/experiments/manual-assignment.md @@ -16,4 +16,4 @@ This prerequisite exercise uses real issue [#10](https://github.com/codeabovelab Codeboost exposed 12 segments before assignment, all in the Unplanned row. Each segment mapped to exactly one item through that item's declared file contract. After the assignments, the per-item segment counts were 4, 4, 2, and 2; all four rows reported `✓ Attributed` and `✓ In scope`. No segment was unclear or multiply owned, so this exercise produced no ambiguity fixture. The earlier rename and literal-path concerns already have focused fixtures in `test/review.test.ts` and `test/plant.test.ts`. -The assignment was rerun from a detached worktree at the exact head using a fresh SQLite review store. The stored choices, rather than a hand-written diff classification alone, produced the final attribution and scope results above. +The assignment was rerun from a detached worktree at the exact head using a fresh SQLite review store. The stored choices, rather than a hand-written diff classification alone, produced the final attribution and scope results above. `manual-assignment.json` records every stable choice ID with its kind, operation, path, original row, assigned item, and final scope so the 12 assignments can be checked individually. diff --git a/docs/experiments/review-summary-edge-cases.md b/docs/experiments/review-summary-edge-cases.md index fc393fb1..29b5ae23 100644 --- a/docs/experiments/review-summary-edge-cases.md +++ b/docs/experiments/review-summary-edge-cases.md @@ -22,4 +22,4 @@ Result: no defect reproduced. The summary supplied no expected color, state toke ## Gate result -All three concerns are resolved as validated behavior. No issue supplied a failing case that justified a production patch. Issue #3's manual assignment and paired human go/no-go experiment remain pending and must use the frozen protocol in `review-protocol.md`. +All three concerns are resolved as validated behavior. No issue supplied a failing case that justified a production patch. Issue #3's manual assignment is recorded in `manual-assignment.md`; its paired human go/no-go experiment remains pending and must use the frozen protocol in `review-protocol.md`. diff --git a/docs/implementation/read-only-review.md b/docs/implementation/read-only-review.md index 1ed2937b..463c6fab 100644 --- a/docs/implementation/read-only-review.md +++ b/docs/implementation/read-only-review.md @@ -22,7 +22,7 @@ Store schema v2 adds the review counter and per-item notes through a transaction - No agent answers, test execution, AI findings ingestion, send-to-agent action, or merge control is included. These belong after the go/no-go gate. Four checks distinguish unavailable evidence from success. - File cards show mode/path/object IDs and blob byte sizes. PNG/JPEG/GIF/WebP previews are bounded to 1 MiB each and 4 MiB across a history; unsupported/oversized images say unavailable. Gitlink byte sizes are not applicable. SVG/HTML is never embedded. -- The protocol at `docs/experiments/review-protocol.md` must be filled with the real pairs and committed before the first timed review. The manual assignment, real paired PRs, human timing, and final result are pending. Do not mark issue #3 closed or claim the gate passed. +- The real issue pairs are frozen in `docs/experiments/review-protocol.md`, and the completed manual assignment is recorded in `docs/experiments/manual-assignment.md`. The paired PRs, exact review revision, human timing, and final result remain pending. Do not mark issue #3 closed or claim the gate passed. - The planting helper is intentionally limited to disposable clones and supported regular top-level paths; it never publishes PRs. ## Validation From b0afccdf65d1d24727adcc7e925c7b55cec0b588 Mon Sep 17 00:00:00 2001 From: mchwang Date: Wed, 23 Sep 2026 18:26:51 -0700 Subject: [PATCH 05/15] docs: record first prepared experiment pair --- docs/experiments/review-protocol.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/experiments/review-protocol.md b/docs/experiments/review-protocol.md index 63a3dbf4..e6e0c35c 100644 --- a/docs/experiments/review-protocol.md +++ b/docs/experiments/review-protocol.md @@ -1,6 +1,6 @@ # Plan-indexed review: go/no-go protocol -Status: **pair selection frozen; implementations and reviews not started**. Max Hwang (`mchwang`) is the blinded reviewer and Codex is the experiment operator. This document does not claim a passed gate. Complete each row's PR and SHA fields and commit that exact revision before the first timed review. No issue, assignment, order, or threshold may change once the first review begins. +Status: **pair selection frozen; pair 1 prepared; reviews not started**. Max Hwang (`mchwang`) is the blinded reviewer and Codex is the experiment operator. This document does not claim a passed gate. Complete each row's PR and SHA fields and commit that exact revision before the first timed review. No issue, assignment, order, or threshold may change once the first review begins. ## Pair selection and order @@ -10,7 +10,7 @@ All eight issues are in `codeabovelab/guardyx-gstack`. A/B placement within each | Pair | Real issue A / PR / base..head | Real issue B / PR / base..head | A method | B method | |---|---|---|---|---| -| 1 — companion documentation | [#493](https://github.com/codeabovelab/guardyx-gstack/issues/493) / PR pending / SHA pending | [#176](https://github.com/codeabovelab/guardyx-gstack/issues/176) / PR pending / SHA pending | Raw GitHub diff | codeboost | +| 1 — companion documentation | [#493](https://github.com/codeabovelab/guardyx-gstack/issues/493) / [PR #600](https://github.com/codeabovelab/guardyx-gstack/pull/600) / `f95a7eba..6a273dab` | [#176](https://github.com/codeabovelab/guardyx-gstack/issues/176) / [PR #601](https://github.com/codeabovelab/guardyx-gstack/pull/601) / `f95a7eba..f61a793e` | Raw GitHub diff | codeboost | | 2 — agent-key security | [#506](https://github.com/codeabovelab/guardyx-gstack/issues/506) / PR pending / SHA pending | [#443](https://github.com/codeabovelab/guardyx-gstack/issues/443) / PR pending / SHA pending | codeboost | Raw GitHub diff | | 3 — asynchronous notifications | [#13](https://github.com/codeabovelab/guardyx-gstack/issues/13) / PR pending / SHA pending | [#429](https://github.com/codeabovelab/guardyx-gstack/issues/429) / PR pending / SHA pending | Raw GitHub diff | codeboost | | 4 — compliance evidence | [#504](https://github.com/codeabovelab/guardyx-gstack/issues/504) / PR pending / SHA pending | [#507](https://github.com/codeabovelab/guardyx-gstack/issues/507) / PR pending / SHA pending | codeboost | Raw GitHub diff | @@ -43,4 +43,4 @@ Report declared-file catch rate for both methods without a pass threshold. Stop ## Current evidence -Engineering tests prove browser interactions, persistence, attribution, and planting mechanics. The separate manual assignment proves that one real merged issue can be completely classified through stored review choices. Neither establishes human review speed or catch rates. Four Guardyx issue pairs are selected, but no paired implementation, timed decision, or catch-rate result exists yet. +Engineering tests prove browser interactions, persistence, attribution, and planting mechanics. The separate manual assignment proves that one real merged issue can be completely classified through stored review choices. Neither establishes human review speed or catch rates. Pair 1 is prepared at the exact PR revisions above and passed the repository preflight; the other three pairs remain in preparation. No timed decision or catch-rate result exists yet. From c8202bc136a313e6a893b689df7bb52bf4ea4945 Mon Sep 17 00:00:00 2001 From: mchwang Date: Wed, 23 Sep 2026 18:31:05 -0700 Subject: [PATCH 06/15] Record exact pair-one revisions --- docs/experiments/review-protocol.md | 2 +- docs/implementation/read-only-review.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/experiments/review-protocol.md b/docs/experiments/review-protocol.md index e6e0c35c..758bf063 100644 --- a/docs/experiments/review-protocol.md +++ b/docs/experiments/review-protocol.md @@ -10,7 +10,7 @@ All eight issues are in `codeabovelab/guardyx-gstack`. A/B placement within each | Pair | Real issue A / PR / base..head | Real issue B / PR / base..head | A method | B method | |---|---|---|---|---| -| 1 — companion documentation | [#493](https://github.com/codeabovelab/guardyx-gstack/issues/493) / [PR #600](https://github.com/codeabovelab/guardyx-gstack/pull/600) / `f95a7eba..6a273dab` | [#176](https://github.com/codeabovelab/guardyx-gstack/issues/176) / [PR #601](https://github.com/codeabovelab/guardyx-gstack/pull/601) / `f95a7eba..f61a793e` | Raw GitHub diff | codeboost | +| 1 — companion documentation | [#493](https://github.com/codeabovelab/guardyx-gstack/issues/493) / [PR #600](https://github.com/codeabovelab/guardyx-gstack/pull/600) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..6a273dab42005e01a6f4c2ea92297f40ad29e0d7` | [#176](https://github.com/codeabovelab/guardyx-gstack/issues/176) / [PR #601](https://github.com/codeabovelab/guardyx-gstack/pull/601) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..f61a793ebe4e7352f7ebf74238adfad3c18750d0` | Raw GitHub diff | codeboost | | 2 — agent-key security | [#506](https://github.com/codeabovelab/guardyx-gstack/issues/506) / PR pending / SHA pending | [#443](https://github.com/codeabovelab/guardyx-gstack/issues/443) / PR pending / SHA pending | codeboost | Raw GitHub diff | | 3 — asynchronous notifications | [#13](https://github.com/codeabovelab/guardyx-gstack/issues/13) / PR pending / SHA pending | [#429](https://github.com/codeabovelab/guardyx-gstack/issues/429) / PR pending / SHA pending | Raw GitHub diff | codeboost | | 4 — compliance evidence | [#504](https://github.com/codeabovelab/guardyx-gstack/issues/504) / PR pending / SHA pending | [#507](https://github.com/codeabovelab/guardyx-gstack/issues/507) / PR pending / SHA pending | codeboost | Raw GitHub diff | diff --git a/docs/implementation/read-only-review.md b/docs/implementation/read-only-review.md index 463c6fab..d71eb718 100644 --- a/docs/implementation/read-only-review.md +++ b/docs/implementation/read-only-review.md @@ -22,7 +22,7 @@ Store schema v2 adds the review counter and per-item notes through a transaction - No agent answers, test execution, AI findings ingestion, send-to-agent action, or merge control is included. These belong after the go/no-go gate. Four checks distinguish unavailable evidence from success. - File cards show mode/path/object IDs and blob byte sizes. PNG/JPEG/GIF/WebP previews are bounded to 1 MiB each and 4 MiB across a history; unsupported/oversized images say unavailable. Gitlink byte sizes are not applicable. SVG/HTML is never embedded. -- The real issue pairs are frozen in `docs/experiments/review-protocol.md`, and the completed manual assignment is recorded in `docs/experiments/manual-assignment.md`. The paired PRs, exact review revision, human timing, and final result remain pending. Do not mark issue #3 closed or claim the gate passed. +- The real issue pairs are frozen in `docs/experiments/review-protocol.md`, and the completed manual assignment is recorded in `docs/experiments/manual-assignment.md`. Pair 1 is prepared at its exact PR revisions; pairs 2–4, human timing, and the final result remain pending. Do not mark issue #3 closed or claim the gate passed. - The planting helper is intentionally limited to disposable clones and supported regular top-level paths; it never publishes PRs. ## Validation From b6d156bb04332b511fd1289fee5e94f93c42d2e9 Mon Sep 17 00:00:00 2001 From: mchwang Date: Wed, 23 Sep 2026 18:53:11 -0700 Subject: [PATCH 07/15] Record issue 506 experiment PR --- docs/experiments/review-protocol.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/experiments/review-protocol.md b/docs/experiments/review-protocol.md index 758bf063..deb407b5 100644 --- a/docs/experiments/review-protocol.md +++ b/docs/experiments/review-protocol.md @@ -11,7 +11,7 @@ All eight issues are in `codeabovelab/guardyx-gstack`. A/B placement within each | Pair | Real issue A / PR / base..head | Real issue B / PR / base..head | A method | B method | |---|---|---|---|---| | 1 — companion documentation | [#493](https://github.com/codeabovelab/guardyx-gstack/issues/493) / [PR #600](https://github.com/codeabovelab/guardyx-gstack/pull/600) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..6a273dab42005e01a6f4c2ea92297f40ad29e0d7` | [#176](https://github.com/codeabovelab/guardyx-gstack/issues/176) / [PR #601](https://github.com/codeabovelab/guardyx-gstack/pull/601) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..f61a793ebe4e7352f7ebf74238adfad3c18750d0` | Raw GitHub diff | codeboost | -| 2 — agent-key security | [#506](https://github.com/codeabovelab/guardyx-gstack/issues/506) / PR pending / SHA pending | [#443](https://github.com/codeabovelab/guardyx-gstack/issues/443) / PR pending / SHA pending | codeboost | Raw GitHub diff | +| 2 — agent-key security | [#506](https://github.com/codeabovelab/guardyx-gstack/issues/506) / [PR #602](https://github.com/codeabovelab/guardyx-gstack/pull/602) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..11a5d74d9bcdd0dcd1faa96e02cfe101ed9c8b6a` | [#443](https://github.com/codeabovelab/guardyx-gstack/issues/443) / PR pending / SHA pending | codeboost | Raw GitHub diff | | 3 — asynchronous notifications | [#13](https://github.com/codeabovelab/guardyx-gstack/issues/13) / PR pending / SHA pending | [#429](https://github.com/codeabovelab/guardyx-gstack/issues/429) / PR pending / SHA pending | Raw GitHub diff | codeboost | | 4 — compliance evidence | [#504](https://github.com/codeabovelab/guardyx-gstack/issues/504) / PR pending / SHA pending | [#507](https://github.com/codeabovelab/guardyx-gstack/issues/507) / PR pending / SHA pending | codeboost | Raw GitHub diff | From bbf2e725bef11b09006a507e5b0a536330bcb333 Mon Sep 17 00:00:00 2001 From: mchwang Date: Wed, 23 Sep 2026 19:08:13 -0700 Subject: [PATCH 08/15] Record raw agent key review PR --- docs/experiments/review-protocol.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/experiments/review-protocol.md b/docs/experiments/review-protocol.md index deb407b5..2e1d3edf 100644 --- a/docs/experiments/review-protocol.md +++ b/docs/experiments/review-protocol.md @@ -11,7 +11,7 @@ All eight issues are in `codeabovelab/guardyx-gstack`. A/B placement within each | Pair | Real issue A / PR / base..head | Real issue B / PR / base..head | A method | B method | |---|---|---|---|---| | 1 — companion documentation | [#493](https://github.com/codeabovelab/guardyx-gstack/issues/493) / [PR #600](https://github.com/codeabovelab/guardyx-gstack/pull/600) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..6a273dab42005e01a6f4c2ea92297f40ad29e0d7` | [#176](https://github.com/codeabovelab/guardyx-gstack/issues/176) / [PR #601](https://github.com/codeabovelab/guardyx-gstack/pull/601) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..f61a793ebe4e7352f7ebf74238adfad3c18750d0` | Raw GitHub diff | codeboost | -| 2 — agent-key security | [#506](https://github.com/codeabovelab/guardyx-gstack/issues/506) / [PR #602](https://github.com/codeabovelab/guardyx-gstack/pull/602) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..11a5d74d9bcdd0dcd1faa96e02cfe101ed9c8b6a` | [#443](https://github.com/codeabovelab/guardyx-gstack/issues/443) / PR pending / SHA pending | codeboost | Raw GitHub diff | +| 2 — agent-key security | [#506](https://github.com/codeabovelab/guardyx-gstack/issues/506) / [PR #602](https://github.com/codeabovelab/guardyx-gstack/pull/602) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..11a5d74d9bcdd0dcd1faa96e02cfe101ed9c8b6a` | [#443](https://github.com/codeabovelab/guardyx-gstack/issues/443) / [PR #603](https://github.com/codeabovelab/guardyx-gstack/pull/603) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..042999d9ebc3dc61fb029b21644bd20b4537a347` | codeboost | Raw GitHub diff | | 3 — asynchronous notifications | [#13](https://github.com/codeabovelab/guardyx-gstack/issues/13) / PR pending / SHA pending | [#429](https://github.com/codeabovelab/guardyx-gstack/issues/429) / PR pending / SHA pending | Raw GitHub diff | codeboost | | 4 — compliance evidence | [#504](https://github.com/codeabovelab/guardyx-gstack/issues/504) / PR pending / SHA pending | [#507](https://github.com/codeabovelab/guardyx-gstack/issues/507) / PR pending / SHA pending | codeboost | Raw GitHub diff | From 5e1bf76211b6f6c453b0c0c0d4801fff3afb044e Mon Sep 17 00:00:00 2001 From: mchwang Date: Wed, 23 Sep 2026 19:36:39 -0700 Subject: [PATCH 09/15] Freeze issue 13 review artifact --- docs/experiments/review-protocol.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/experiments/review-protocol.md b/docs/experiments/review-protocol.md index 2e1d3edf..a8334754 100644 --- a/docs/experiments/review-protocol.md +++ b/docs/experiments/review-protocol.md @@ -12,7 +12,7 @@ All eight issues are in `codeabovelab/guardyx-gstack`. A/B placement within each |---|---|---|---|---| | 1 — companion documentation | [#493](https://github.com/codeabovelab/guardyx-gstack/issues/493) / [PR #600](https://github.com/codeabovelab/guardyx-gstack/pull/600) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..6a273dab42005e01a6f4c2ea92297f40ad29e0d7` | [#176](https://github.com/codeabovelab/guardyx-gstack/issues/176) / [PR #601](https://github.com/codeabovelab/guardyx-gstack/pull/601) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..f61a793ebe4e7352f7ebf74238adfad3c18750d0` | Raw GitHub diff | codeboost | | 2 — agent-key security | [#506](https://github.com/codeabovelab/guardyx-gstack/issues/506) / [PR #602](https://github.com/codeabovelab/guardyx-gstack/pull/602) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..11a5d74d9bcdd0dcd1faa96e02cfe101ed9c8b6a` | [#443](https://github.com/codeabovelab/guardyx-gstack/issues/443) / [PR #603](https://github.com/codeabovelab/guardyx-gstack/pull/603) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..042999d9ebc3dc61fb029b21644bd20b4537a347` | codeboost | Raw GitHub diff | -| 3 — asynchronous notifications | [#13](https://github.com/codeabovelab/guardyx-gstack/issues/13) / PR pending / SHA pending | [#429](https://github.com/codeabovelab/guardyx-gstack/issues/429) / PR pending / SHA pending | Raw GitHub diff | codeboost | +| 3 — asynchronous notifications | [#13](https://github.com/codeabovelab/guardyx-gstack/issues/13) / [PR #604](https://github.com/codeabovelab/guardyx-gstack/pull/604) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..6265ec3d7e0a023972000626a7309dbbf3c941cb` | [#429](https://github.com/codeabovelab/guardyx-gstack/issues/429) / PR pending / SHA pending | Raw GitHub diff | codeboost | | 4 — compliance evidence | [#504](https://github.com/codeabovelab/guardyx-gstack/issues/504) / PR pending / SHA pending | [#507](https://github.com/codeabovelab/guardyx-gstack/issues/507) / PR pending / SHA pending | codeboost | Raw GitHub diff | The pairs are matched before implementation on repository, primary implementation language, scope, and expected review surface. Pair 1 changes operational/compliance documentation without runtime behavior; pair 2 changes agent-key lifecycle across the API and UI; pair 3 adds medium asynchronous notification delivery across backend jobs and consumer-facing interfaces; pair 4 adds compliance evidence across stored invocation data, exports, mapping data, and PDF surfaces. If implementation discovery makes a pair materially unequal or blocked, stop before the first timed review and record a new frozen protocol revision rather than silently substituting an issue. From 7751453d0cd9e412192adafbb63184334176f245 Mon Sep 17 00:00:00 2001 From: mchwang Date: Wed, 23 Sep 2026 20:02:54 -0700 Subject: [PATCH 10/15] Record issue 429 review package --- docs/experiments/review-protocol.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/experiments/review-protocol.md b/docs/experiments/review-protocol.md index a8334754..9ce57ea5 100644 --- a/docs/experiments/review-protocol.md +++ b/docs/experiments/review-protocol.md @@ -12,7 +12,7 @@ All eight issues are in `codeabovelab/guardyx-gstack`. A/B placement within each |---|---|---|---|---| | 1 — companion documentation | [#493](https://github.com/codeabovelab/guardyx-gstack/issues/493) / [PR #600](https://github.com/codeabovelab/guardyx-gstack/pull/600) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..6a273dab42005e01a6f4c2ea92297f40ad29e0d7` | [#176](https://github.com/codeabovelab/guardyx-gstack/issues/176) / [PR #601](https://github.com/codeabovelab/guardyx-gstack/pull/601) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..f61a793ebe4e7352f7ebf74238adfad3c18750d0` | Raw GitHub diff | codeboost | | 2 — agent-key security | [#506](https://github.com/codeabovelab/guardyx-gstack/issues/506) / [PR #602](https://github.com/codeabovelab/guardyx-gstack/pull/602) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..11a5d74d9bcdd0dcd1faa96e02cfe101ed9c8b6a` | [#443](https://github.com/codeabovelab/guardyx-gstack/issues/443) / [PR #603](https://github.com/codeabovelab/guardyx-gstack/pull/603) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..042999d9ebc3dc61fb029b21644bd20b4537a347` | codeboost | Raw GitHub diff | -| 3 — asynchronous notifications | [#13](https://github.com/codeabovelab/guardyx-gstack/issues/13) / [PR #604](https://github.com/codeabovelab/guardyx-gstack/pull/604) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..6265ec3d7e0a023972000626a7309dbbf3c941cb` | [#429](https://github.com/codeabovelab/guardyx-gstack/issues/429) / PR pending / SHA pending | Raw GitHub diff | codeboost | +| 3 — asynchronous notifications | [#13](https://github.com/codeabovelab/guardyx-gstack/issues/13) / [PR #604](https://github.com/codeabovelab/guardyx-gstack/pull/604) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..6265ec3d7e0a023972000626a7309dbbf3c941cb` | [#429](https://github.com/codeabovelab/guardyx-gstack/issues/429) / [PR #605](https://github.com/codeabovelab/guardyx-gstack/pull/605) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..66a2dc798e5d1b187fd9110cf8df2b66e27ee489` | Raw GitHub diff | codeboost | | 4 — compliance evidence | [#504](https://github.com/codeabovelab/guardyx-gstack/issues/504) / PR pending / SHA pending | [#507](https://github.com/codeabovelab/guardyx-gstack/issues/507) / PR pending / SHA pending | codeboost | Raw GitHub diff | The pairs are matched before implementation on repository, primary implementation language, scope, and expected review surface. Pair 1 changes operational/compliance documentation without runtime behavior; pair 2 changes agent-key lifecycle across the API and UI; pair 3 adds medium asynchronous notification delivery across backend jobs and consumer-facing interfaces; pair 4 adds compliance evidence across stored invocation data, exports, mapping data, and PDF surfaces. If implementation discovery makes a pair materially unequal or blocked, stop before the first timed review and record a new frozen protocol revision rather than silently substituting an issue. From e83d8b32115ac0f3a4cd30b7b4d5395a7572ff91 Mon Sep 17 00:00:00 2001 From: mchwang Date: Wed, 23 Sep 2026 20:24:38 -0700 Subject: [PATCH 11/15] Record issue 504 experiment package --- docs/experiments/review-protocol.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/experiments/review-protocol.md b/docs/experiments/review-protocol.md index 9ce57ea5..40d73fd0 100644 --- a/docs/experiments/review-protocol.md +++ b/docs/experiments/review-protocol.md @@ -13,7 +13,7 @@ All eight issues are in `codeabovelab/guardyx-gstack`. A/B placement within each | 1 — companion documentation | [#493](https://github.com/codeabovelab/guardyx-gstack/issues/493) / [PR #600](https://github.com/codeabovelab/guardyx-gstack/pull/600) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..6a273dab42005e01a6f4c2ea92297f40ad29e0d7` | [#176](https://github.com/codeabovelab/guardyx-gstack/issues/176) / [PR #601](https://github.com/codeabovelab/guardyx-gstack/pull/601) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..f61a793ebe4e7352f7ebf74238adfad3c18750d0` | Raw GitHub diff | codeboost | | 2 — agent-key security | [#506](https://github.com/codeabovelab/guardyx-gstack/issues/506) / [PR #602](https://github.com/codeabovelab/guardyx-gstack/pull/602) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..11a5d74d9bcdd0dcd1faa96e02cfe101ed9c8b6a` | [#443](https://github.com/codeabovelab/guardyx-gstack/issues/443) / [PR #603](https://github.com/codeabovelab/guardyx-gstack/pull/603) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..042999d9ebc3dc61fb029b21644bd20b4537a347` | codeboost | Raw GitHub diff | | 3 — asynchronous notifications | [#13](https://github.com/codeabovelab/guardyx-gstack/issues/13) / [PR #604](https://github.com/codeabovelab/guardyx-gstack/pull/604) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..6265ec3d7e0a023972000626a7309dbbf3c941cb` | [#429](https://github.com/codeabovelab/guardyx-gstack/issues/429) / [PR #605](https://github.com/codeabovelab/guardyx-gstack/pull/605) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..66a2dc798e5d1b187fd9110cf8df2b66e27ee489` | Raw GitHub diff | codeboost | -| 4 — compliance evidence | [#504](https://github.com/codeabovelab/guardyx-gstack/issues/504) / PR pending / SHA pending | [#507](https://github.com/codeabovelab/guardyx-gstack/issues/507) / PR pending / SHA pending | codeboost | Raw GitHub diff | +| 4 — compliance evidence | [#504](https://github.com/codeabovelab/guardyx-gstack/issues/504) / [PR #606](https://github.com/codeabovelab/guardyx-gstack/pull/606) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..7a084a39a4bbbd0d3d78796407ec391acbce6095` | [#507](https://github.com/codeabovelab/guardyx-gstack/issues/507) / PR pending / SHA pending | codeboost | Raw GitHub diff | The pairs are matched before implementation on repository, primary implementation language, scope, and expected review surface. Pair 1 changes operational/compliance documentation without runtime behavior; pair 2 changes agent-key lifecycle across the API and UI; pair 3 adds medium asynchronous notification delivery across backend jobs and consumer-facing interfaces; pair 4 adds compliance evidence across stored invocation data, exports, mapping data, and PDF surfaces. If implementation discovery makes a pair materially unequal or blocked, stop before the first timed review and record a new frozen protocol revision rather than silently substituting an issue. From a028706f76d33e4c01cdd384cad758f552c8f36b Mon Sep 17 00:00:00 2001 From: mchwang Date: Wed, 23 Sep 2026 21:04:52 -0700 Subject: [PATCH 12/15] Record final review experiment package --- docs/experiments/review-protocol.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/experiments/review-protocol.md b/docs/experiments/review-protocol.md index 40d73fd0..1a3a7ba8 100644 --- a/docs/experiments/review-protocol.md +++ b/docs/experiments/review-protocol.md @@ -13,7 +13,7 @@ All eight issues are in `codeabovelab/guardyx-gstack`. A/B placement within each | 1 — companion documentation | [#493](https://github.com/codeabovelab/guardyx-gstack/issues/493) / [PR #600](https://github.com/codeabovelab/guardyx-gstack/pull/600) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..6a273dab42005e01a6f4c2ea92297f40ad29e0d7` | [#176](https://github.com/codeabovelab/guardyx-gstack/issues/176) / [PR #601](https://github.com/codeabovelab/guardyx-gstack/pull/601) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..f61a793ebe4e7352f7ebf74238adfad3c18750d0` | Raw GitHub diff | codeboost | | 2 — agent-key security | [#506](https://github.com/codeabovelab/guardyx-gstack/issues/506) / [PR #602](https://github.com/codeabovelab/guardyx-gstack/pull/602) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..11a5d74d9bcdd0dcd1faa96e02cfe101ed9c8b6a` | [#443](https://github.com/codeabovelab/guardyx-gstack/issues/443) / [PR #603](https://github.com/codeabovelab/guardyx-gstack/pull/603) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..042999d9ebc3dc61fb029b21644bd20b4537a347` | codeboost | Raw GitHub diff | | 3 — asynchronous notifications | [#13](https://github.com/codeabovelab/guardyx-gstack/issues/13) / [PR #604](https://github.com/codeabovelab/guardyx-gstack/pull/604) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..6265ec3d7e0a023972000626a7309dbbf3c941cb` | [#429](https://github.com/codeabovelab/guardyx-gstack/issues/429) / [PR #605](https://github.com/codeabovelab/guardyx-gstack/pull/605) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..66a2dc798e5d1b187fd9110cf8df2b66e27ee489` | Raw GitHub diff | codeboost | -| 4 — compliance evidence | [#504](https://github.com/codeabovelab/guardyx-gstack/issues/504) / [PR #606](https://github.com/codeabovelab/guardyx-gstack/pull/606) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..7a084a39a4bbbd0d3d78796407ec391acbce6095` | [#507](https://github.com/codeabovelab/guardyx-gstack/issues/507) / PR pending / SHA pending | codeboost | Raw GitHub diff | +| 4 — compliance evidence | [#504](https://github.com/codeabovelab/guardyx-gstack/issues/504) / [PR #606](https://github.com/codeabovelab/guardyx-gstack/pull/606) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..7a084a39a4bbbd0d3d78796407ec391acbce6095` | [#507](https://github.com/codeabovelab/guardyx-gstack/issues/507) / [PR #607](https://github.com/codeabovelab/guardyx-gstack/pull/607) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..0cc0096adeae699c6172fa7c9cb6e851aedd96ad` | codeboost | Raw GitHub diff | The pairs are matched before implementation on repository, primary implementation language, scope, and expected review surface. Pair 1 changes operational/compliance documentation without runtime behavior; pair 2 changes agent-key lifecycle across the API and UI; pair 3 adds medium asynchronous notification delivery across backend jobs and consumer-facing interfaces; pair 4 adds compliance evidence across stored invocation data, exports, mapping data, and PDF surfaces. If implementation discovery makes a pair materially unequal or blocked, stop before the first timed review and record a new frozen protocol revision rather than silently substituting an issue. From 645880eabc0bafea7249a93cf93da4478fdb683f Mon Sep 17 00:00:00 2001 From: mchwang Date: Wed, 23 Sep 2026 21:06:03 -0700 Subject: [PATCH 13/15] Mark all review experiment pairs prepared --- docs/experiments/review-protocol.md | 4 ++-- docs/implementation/read-only-review.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/experiments/review-protocol.md b/docs/experiments/review-protocol.md index 1a3a7ba8..5f7b457f 100644 --- a/docs/experiments/review-protocol.md +++ b/docs/experiments/review-protocol.md @@ -1,6 +1,6 @@ # Plan-indexed review: go/no-go protocol -Status: **pair selection frozen; pair 1 prepared; reviews not started**. Max Hwang (`mchwang`) is the blinded reviewer and Codex is the experiment operator. This document does not claim a passed gate. Complete each row's PR and SHA fields and commit that exact revision before the first timed review. No issue, assignment, order, or threshold may change once the first review begins. +Status: **pair selection frozen; all four pairs prepared; reviews not started**. Max Hwang (`mchwang`) is the blinded reviewer and Codex is the experiment operator. This document does not claim a passed gate. Every row's PR and exact base/head SHAs are committed below. No issue, assignment, order, or threshold may change once the first review begins. ## Pair selection and order @@ -43,4 +43,4 @@ Report declared-file catch rate for both methods without a pass threshold. Stop ## Current evidence -Engineering tests prove browser interactions, persistence, attribution, and planting mechanics. The separate manual assignment proves that one real merged issue can be completely classified through stored review choices. Neither establishes human review speed or catch rates. Pair 1 is prepared at the exact PR revisions above and passed the repository preflight; the other three pairs remain in preparation. No timed decision or catch-rate result exists yet. +Engineering tests prove browser interactions, persistence, attribution, and planting mechanics. The separate manual assignment proves that one real merged issue can be completely classified through stored review choices. Neither establishes human review speed or catch rates. All four pairs are prepared at the eight exact PR revisions above, and every revision passed the repository preflight. No timed decision or catch-rate result exists yet. diff --git a/docs/implementation/read-only-review.md b/docs/implementation/read-only-review.md index d71eb718..cfb18f32 100644 --- a/docs/implementation/read-only-review.md +++ b/docs/implementation/read-only-review.md @@ -22,7 +22,7 @@ Store schema v2 adds the review counter and per-item notes through a transaction - No agent answers, test execution, AI findings ingestion, send-to-agent action, or merge control is included. These belong after the go/no-go gate. Four checks distinguish unavailable evidence from success. - File cards show mode/path/object IDs and blob byte sizes. PNG/JPEG/GIF/WebP previews are bounded to 1 MiB each and 4 MiB across a history; unsupported/oversized images say unavailable. Gitlink byte sizes are not applicable. SVG/HTML is never embedded. -- The real issue pairs are frozen in `docs/experiments/review-protocol.md`, and the completed manual assignment is recorded in `docs/experiments/manual-assignment.md`. Pair 1 is prepared at its exact PR revisions; pairs 2–4, human timing, and the final result remain pending. Do not mark issue #3 closed or claim the gate passed. +- The real issue pairs are frozen in `docs/experiments/review-protocol.md`, and the completed manual assignment is recorded in `docs/experiments/manual-assignment.md`. All four pairs are prepared at their exact PR revisions; human timing and the final result remain pending. Do not mark issue #3 closed or claim the gate passed. - The planting helper is intentionally limited to disposable clones and supported regular top-level paths; it never publishes PRs. ## Validation From 1568cf3adeb3cbd8d14128f0a06f8b3299aa3260 Mon Sep 17 00:00:00 2001 From: mchwang Date: Wed, 23 Sep 2026 22:26:04 -0700 Subject: [PATCH 14/15] Replace unblinded experiment packages --- docs/experiments/review-protocol.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/experiments/review-protocol.md b/docs/experiments/review-protocol.md index 5f7b457f..8c877bd0 100644 --- a/docs/experiments/review-protocol.md +++ b/docs/experiments/review-protocol.md @@ -12,13 +12,15 @@ All eight issues are in `codeabovelab/guardyx-gstack`. A/B placement within each |---|---|---|---|---| | 1 — companion documentation | [#493](https://github.com/codeabovelab/guardyx-gstack/issues/493) / [PR #600](https://github.com/codeabovelab/guardyx-gstack/pull/600) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..6a273dab42005e01a6f4c2ea92297f40ad29e0d7` | [#176](https://github.com/codeabovelab/guardyx-gstack/issues/176) / [PR #601](https://github.com/codeabovelab/guardyx-gstack/pull/601) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..f61a793ebe4e7352f7ebf74238adfad3c18750d0` | Raw GitHub diff | codeboost | | 2 — agent-key security | [#506](https://github.com/codeabovelab/guardyx-gstack/issues/506) / [PR #602](https://github.com/codeabovelab/guardyx-gstack/pull/602) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..11a5d74d9bcdd0dcd1faa96e02cfe101ed9c8b6a` | [#443](https://github.com/codeabovelab/guardyx-gstack/issues/443) / [PR #603](https://github.com/codeabovelab/guardyx-gstack/pull/603) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..042999d9ebc3dc61fb029b21644bd20b4537a347` | codeboost | Raw GitHub diff | -| 3 — asynchronous notifications | [#13](https://github.com/codeabovelab/guardyx-gstack/issues/13) / [PR #604](https://github.com/codeabovelab/guardyx-gstack/pull/604) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..6265ec3d7e0a023972000626a7309dbbf3c941cb` | [#429](https://github.com/codeabovelab/guardyx-gstack/issues/429) / [PR #605](https://github.com/codeabovelab/guardyx-gstack/pull/605) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..66a2dc798e5d1b187fd9110cf8df2b66e27ee489` | Raw GitHub diff | codeboost | -| 4 — compliance evidence | [#504](https://github.com/codeabovelab/guardyx-gstack/issues/504) / [PR #606](https://github.com/codeabovelab/guardyx-gstack/pull/606) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..7a084a39a4bbbd0d3d78796407ec391acbce6095` | [#507](https://github.com/codeabovelab/guardyx-gstack/issues/507) / [PR #607](https://github.com/codeabovelab/guardyx-gstack/pull/607) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..0cc0096adeae699c6172fa7c9cb6e851aedd96ad` | codeboost | Raw GitHub diff | +| 3 — asynchronous notifications | [#13](https://github.com/codeabovelab/guardyx-gstack/issues/13) / [PR #608](https://github.com/codeabovelab/guardyx-gstack/pull/608) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..78f62ee413a291a5f0ea555c39bc8d61b6721f9b` | [#429](https://github.com/codeabovelab/guardyx-gstack/issues/429) / [PR #609](https://github.com/codeabovelab/guardyx-gstack/pull/609) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..b8cd873ac688fa27ac841fb2e2df2d6d135446c6` | Raw GitHub diff | codeboost | +| 4 — compliance evidence | [#504](https://github.com/codeabovelab/guardyx-gstack/issues/504) / [PR #610](https://github.com/codeabovelab/guardyx-gstack/pull/610) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..4ce1259c396ed40001efc02216a4cd181f8e50b4` | [#507](https://github.com/codeabovelab/guardyx-gstack/issues/507) / [PR #611](https://github.com/codeabovelab/guardyx-gstack/pull/611) / `f95a7ebaaa1e48a81fa724e8d6be677a17e7ee69..f9988770b89a14f006adf0ecefb15115d3d748d0` | codeboost | Raw GitHub diff | The pairs are matched before implementation on repository, primary implementation language, scope, and expected review surface. Pair 1 changes operational/compliance documentation without runtime behavior; pair 2 changes agent-key lifecycle across the API and UI; pair 3 adds medium asynchronous notification delivery across backend jobs and consumer-facing interfaces; pair 4 adds compliance evidence across stored invocation data, exports, mapping data, and PDF surfaces. If implementation discovery makes a pair materially unequal or blocked, stop before the first timed review and record a new frozen protocol revision rather than silently substituting an issue. Selection history: the first frozen draft paired #554 with #556. Before either implementation began, their required baseline check found that both depend on live 1,400-action/session telemetry unavailable from the operator's environment. A proposed replacement included #335, but inspect-before-build then proved that `main` already contains its requested in-flight guard even though the issue remains open. A current-tree audit also excluded other stale open issues whose requested work has shipped. The table above contains only work confirmed absent from current `main`; every A/B placement was randomized after that audit. No timed review, experimental implementation, or plant had started before this final selection. +Before timed review began, an automated reviewer was mistakenly requested on the original pair 3 and pair 4 draft PRs. That invalidated reviewer blindness for those four packages. PRs #604–#607 were closed, fresh randomized packages were created without changing their issues or method assignments, and the replacement PRs and exact revisions are recorded above. Automated review stays off every experimental draft until its human decision is recorded. + The reviewer must not implement the paired changes or inspect the plants before deciding on the PR. The operator must not expose plant inputs, `sealed.json`, mappings, planted locations, or unblinded implementation notes in the PR, review package, task messages, or protocol updates. An operator prepares plans and commits by hand, records trusted ledger ownership, and supplies both the plan and code to each review method. The earlier one-issue manual assignment is recorded in `manual-assignment.md`; all 12 segments were assigned and in scope, with no unclear case requiring another fixture. ## Planting From f4618ceb2f68af95ec1ed7e302eed24621ca7d92 Mon Sep 17 00:00:00 2001 From: mchwang Date: Wed, 23 Sep 2026 22:33:06 -0700 Subject: [PATCH 15/15] Synchronize experiment prerequisite status --- docs/implementation/build-step-1.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/implementation/build-step-1.md b/docs/implementation/build-step-1.md index dd4d09db..69ccb0b1 100644 --- a/docs/implementation/build-step-1.md +++ b/docs/implementation/build-step-1.md @@ -71,7 +71,7 @@ Fixed aggregate diff retention outside the blob budget. The adapter limits total This is a working foundation, not a completed application or a claim that all implementation tasks are done. T18's pure validation/edit core is present; its agent adapters, import UI, and persistence are pending. Ledger storage, rebase mappings, and the read-only review screen remain next. The already-fixed GitHub check belongs to the later GitHub/runner integration. -The design's manual real-issue assignment and timed go/no-go experiment have not been performed. Disposable Git histories are engineering tests, not evidence that plan-indexed review beats raw review. Write and commit the experiment protocol before using the real review screen for that comparison. Do not proceed to merging, agent execution, planning UI, queue, or learning until the documented gate passes. +The design's manual real-issue assignment is complete, and the timed go/no-go experiment is prepared but has not been performed. Disposable Git histories and the assignment exercise are engineering evidence, not evidence that plan-indexed review beats raw review. Use the frozen experiment protocol for the timed comparison. Do not proceed to merging, agent execution, planning UI, queue, or learning until the documented gate passes. ## Alignment with the merged v1 contract (#6)