-
Notifications
You must be signed in to change notification settings - Fork 0
Freeze paired review experiment inputs #18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
8a226ea
Freeze paired review experiment inputs
mchwang a36162c
Replace telemetry-blocked experiment pair
mchwang d7f9b11
Verify experiment issues against current main
mchwang 576b973
Make manual assignment evidence reproducible
mchwang b0afccd
docs: record first prepared experiment pair
mchwang c8202bc
Record exact pair-one revisions
mchwang b6d156b
Record issue 506 experiment PR
mchwang bbf2e72
Record raw agent key review PR
mchwang 5e1bf76
Freeze issue 13 review artifact
mchwang 7751453
Record issue 429 review package
mchwang e83d8b3
Record issue 504 experiment package
mchwang a028706
Record final review experiment package
mchwang 645880e
Mark all review experiment pairs prepared
mchwang 1568cf3
Replace unblinded experiment packages
mchwang f4618ce
Synchronize experiment prerequisite status
mchwang File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| { | ||
| "base": "5a2685cc709ca8f9f384e35e4d733012c7290d69", | ||
| "head": "442147d1bc9a33edeca1a605c9de3ada13f397f1", | ||
| "segments": [ | ||
| { "choice_id": "96c76cee6518b177f272d8dba1f8db83314d0c09b62f036c6d6f41de40227d5c", "kind": "text", "operation": "+", "path": "AGENTS.md", "before_row": "Unplanned", "item": "P4", "scope": "in-scope" }, | ||
| { "choice_id": "6000e022cb5f2c7fb602ba9dbd6b590cdf0672b1bdc984c036bf563423a86afd", "kind": "text", "operation": "+", "path": "AGENTS.md", "before_row": "Unplanned", "item": "P4", "scope": "in-scope" }, | ||
| { "choice_id": "aa5dbe642c11d5f5f6f1630c0f979006b8e0e3182da3e0aeee8485fa997784d1", "kind": "file", "operation": null, "path": "docs/experiments/review-summary-edge-cases.md", "before_row": "Unplanned", "item": "P3", "scope": "in-scope" }, | ||
| { "choice_id": "dfcf7097d83b6d55df0e4959f111370a96976b1c25f327e9d51838fc8224ceaf", "kind": "text", "operation": "+", "path": "docs/experiments/review-summary-edge-cases.md", "before_row": "Unplanned", "item": "P3", "scope": "in-scope" }, | ||
| { "choice_id": "7a5227f9a6be94d6c16d1ea058ae3a5d956a5b53f93333a15d3b778222a90218", "kind": "text", "operation": "-", "path": "test/plant.test.ts", "before_row": "Unplanned", "item": "P2", "scope": "in-scope" }, | ||
| { "choice_id": "142cf07bee22dbe7cf281b0d97efa49f9a502fed62701de8ec0c94500a94c788", "kind": "text", "operation": "+", "path": "test/plant.test.ts", "before_row": "Unplanned", "item": "P2", "scope": "in-scope" }, | ||
| { "choice_id": "36cb67aa87587c9159f19ce0bac78a4d67f1348f55108f99e1fed99e5273a414", "kind": "text", "operation": "+", "path": "test/plant.test.ts", "before_row": "Unplanned", "item": "P2", "scope": "in-scope" }, | ||
| { "choice_id": "88fada2e5b9a307f6db4fa6ccdbbf9e5bfbf00f26cbcacb33ac5bc141d74b85e", "kind": "text", "operation": "+", "path": "test/plant.test.ts", "before_row": "Unplanned", "item": "P2", "scope": "in-scope" }, | ||
| { "choice_id": "9f010f7ae66fd80d72c5d78d45d4effbcdac76ebb6dfadc4efd4e66ac4af9f86", "kind": "text", "operation": "-", "path": "test/review.test.ts", "before_row": "Unplanned", "item": "P1", "scope": "in-scope" }, | ||
| { "choice_id": "987cbec380909ee199bbe4df04a7d700c6f2a1f284c52f08120c159f8e92c35d", "kind": "text", "operation": "+", "path": "test/review.test.ts", "before_row": "Unplanned", "item": "P1", "scope": "in-scope" }, | ||
| { "choice_id": "6b276597aa7294f44e5a549e38d423876266fe6a9a502bb5be089db2223a8976", "kind": "text", "operation": "+", "path": "test/review.test.ts", "before_row": "Unplanned", "item": "P1", "scope": "in-scope" }, | ||
| { "choice_id": "c9379939cb69513a37cbb385440217283975294a2f8b340839d5f9d09ea126df", "kind": "text", "operation": "+", "path": "test/review.test.ts", "before_row": "Unplanned", "item": "P1", "scope": "in-scope" } | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| # Manual real-issue assignment | ||
|
|
||
| This prerequisite exercise uses real issue [#10](https://github.com/codeabovelab/codeboost/issues/10) and merged [PR #17](https://github.com/codeabovelab/codeboost/pull/17). It is an attribution exercise, not a timed trial and not evidence for the go/no-go thresholds. | ||
|
|
||
| - Base: `5a2685cc709ca8f9f384e35e4d733012c7290d69` | ||
| - Head: `442147d1bc9a33edeca1a605c9de3ada13f397f1` | ||
| - Ledger setup: the squash commit was intentionally recorded as foreign so every resulting segment required an explicit manual assignment. | ||
| - Plan: P1 rename-reassignment regression, P2 literal-path regression, P3 evidence record, and P4 durable review rules. | ||
|
|
||
| | Plan item | Declared file | Assigned segments | Final attribution | Final scope | | ||
| |---|---|---:|---|---| | ||
| | P1 | `test/review.test.ts` | 4 text | Attributed | In scope | | ||
| | P2 | `test/plant.test.ts` | 4 text | Attributed | In scope | | ||
| | P3 | `docs/experiments/review-summary-edge-cases.md` | 1 file + 1 text | Attributed | In scope | | ||
| | P4 | `AGENTS.md` | 2 text | Attributed | In scope | | ||
|
|
||
| Codeboost exposed 12 segments before assignment, all in the Unplanned row. Each segment mapped to exactly one item through that item's declared file contract. After the assignments, the per-item segment counts were 4, 4, 2, and 2; all four rows reported `✓ Attributed` and `✓ In scope`. No segment was unclear or multiply owned, so this exercise produced no ambiguity fixture. The earlier rename and literal-path concerns already have focused fixtures in `test/review.test.ts` and `test/plant.test.ts`. | ||
|
|
||
| The assignment was rerun from a detached worktree at the exact head using a fresh SQLite review store. The stored choices, rather than a hand-written diff classification alone, produced the final attribution and scope results above. `manual-assignment.json` records every stable choice ID with its kind, operation, path, original row, assigned item, and final scope so the 12 assignments can be checked individually. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.