Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 3 additions & 16 deletions docs/scenarios/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,46 +7,33 @@ Generated by `bun run scenarios:build`. Edits here are overwritten; change the s

| | conversation | voices | acts |
| --- | --- | --- | --- |
| [S-1](S-1.md) | A hunch that is not yet an experiment | Researcher | 15 |
| [S-1](S-1.md) | A hunch that is not yet an experiment | Researcher | 16 |
| [S-3](S-3.md) | Significant by the primary test, untrustworthy by its own robustness checks | Researcher | 8 |
| [S-3b](S-3b.md) | The same design with nothing downstream | Researcher | 9 |
| [S-3c](S-3c.md) | The check was wrong, not the result | Researcher | 11 |
| [S-4](S-4.md) | a negative result that closes the question | Researcher, Agent | 9 |
| [S-5](S-5.md) | contradiction or dissociation? | Researcher | 10 |
| [S-7](S-7.md) | locked design, then feasibility finds a mechanical defect | Researcher, Agent | 14 |
| [S-8](S-8.md) | don't spend the whole budget discovering the pipeline is broken | Researcher, Agent | 11 |
| [S-9](S-9.md) | the artefact survived; its provenance didn't | Researcher | 7 |
| [S-9](S-9.md) | the artefact survived; its provenance didn't | Researcher | 9 |
| [S-9b](S-9b.md) | was this a rebuild, or new work? | Researcher | 7 |
| [S-9c](S-9c.md) | two parts, one name | Researcher | 5 |
| [S-9d](S-9d.md) | resting on one thing, or two? | Researcher | 5 |
| [S-9e](S-9e.md) | reproducing nothing | Researcher | 3 |
| [S-10](S-10.md) | Rerunning is not reproducing | Researcher | 6 |
| [S-10b](S-10b.md) | The same inputs, in a different order | Researcher | 5 |
| [S-10c](S-10c.md) | Which input changed? | Researcher | 6 |
| [S-10d](S-10d.md) | The order a run read its inputs in | Researcher | 6 |
| [S-10e](S-10e.md) | The same record, read twice by one run | Researcher | 5 |
| [S-11](S-11.md) | The analysis was wrong; the observations were fine | Researcher, Reviewer | 17 |
| [S-11c](S-11c.md) | Nothing found is not nothing there | Researcher | 7 |
| [S-11d](S-11d.md) | A stage cannot read a stage | Researcher | 6 |
| [S-11e](S-11e.md) | A replacement that consumes the output it invalidated | Researcher, Reviewer | 7 |
| [S-11f](S-11f.md) | A computed input, asked about by the reads that touch inputs | Researcher | 6 |
| [S-11g](S-11g.md) | A replacement that addresses only some of a run's conclusions | Researcher, Reviewer | 8 |
| [S-12](S-12.md) | The numbers are right; the sentence about them is wrong | Researcher | 8 |
| [S-12b](S-12b.md) | Two revision chains that pass through one sentence | Researcher | 12 |
| [S-14](S-14.md) | Deliberately leaving something unresolved | Researcher | 5 |
| [S-17](S-17.md) | Does the guard actually guard? | Researcher | 3 |
| [S-18](S-18.md) | Scratch work that unexpectedly mattered | Researcher | 5 |
| [S-18b](S-18b.md) | A negative result that somebody vouched for | Researcher | 6 |
| [S-19](S-19.md) | promoted, closed, and the agreed check never run | Researcher | 7 |
| [S-20](S-20.md) | a finding that settles the proposition neither way | Researcher | 5 |
| [S-21](S-21.md) | a finding drawn across findings | Researcher | 14 |
| [S-22](S-22.md) | a check decided by measurement says so | Researcher | 6 |
| [S-23](S-23.md) | prespecified is not promoted | Researcher | 6 |
| [S-24](S-24.md) | a mistaken act taken back | Researcher | 2 |
| [S-24b](S-24b.md) | walking back a tree of mistakes | Researcher | 12 |
| [S-25](S-25.md) | one rule judged four times | Researcher | 20 |
| [S-26](S-26.md) | work nobody is doing | Researcher | 5 |
| [S-27](S-27.md) | Why explains every kind | Researcher | 11 |
| [S-27](S-27.md) | Why explains every kind | Researcher | 10 |
| [S-28](S-28.md) | A hunch became a question | Researcher | 2 |
| [S-29](S-29.md) | The note came after the question | Researcher | 2 |
| [S-30](S-30.md) | Fixed before the first run | Researcher | 4 |
Expand Down
8 changes: 5 additions & 3 deletions docs/scenarios/S-1.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

A researcher has a vague idea about what a learned topology is doing. Before anything is run, the record says what is already established, what is still open and what has never been tested, and the hunch is narrowed into a question that could be answered.

15 acts, one voice.
16 acts, one voice.

```mermaid
sequenceDiagram
Expand All @@ -26,7 +26,8 @@ sequenceDiagram
R-->>Researcher: COMP_11
Researcher->>R: conclude the internal response is more than a nonlinear smear
Researcher->>R: pose x2
Researcher->>R: sharpen the vague form is not testable this one names what would count …
Researcher->>R: note
Researcher->>R: pose do different inputs map to reproducibly different internal resp…
```

## The acts, in order
Expand All @@ -47,4 +48,5 @@ sequenceDiagram
| 12 | Researcher | `conclude` | the internal response is more than a nonlinear smear | `COMP_11` |
| 13 | Researcher | `pose` | does the learned topology help on an external task? | `Q_13` |
| 14 | Researcher | `pose` | is the learned topology doing something computationally interesting? | `Q_14` |
| 15 | Researcher | `sharpen` | the vague form is not testable; this one names what would count as an answer | `Q_15` |
| 15 | Researcher | `note` | | `NOTE_15` |
| 16 | Researcher | `pose` | do different inputs map to reproducibly different internal responses? | `Q_16` |
26 changes: 0 additions & 26 deletions docs/scenarios/S-10b.md

This file was deleted.

29 changes: 0 additions & 29 deletions docs/scenarios/S-10c.md

This file was deleted.

28 changes: 0 additions & 28 deletions docs/scenarios/S-10d.md

This file was deleted.

27 changes: 0 additions & 27 deletions docs/scenarios/S-10e.md

This file was deleted.

21 changes: 9 additions & 12 deletions docs/scenarios/S-11.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# S-11 — The analysis was wrong; the observations were fine

A reviewer finds the analysis does not implement the null it claims. The observations stand; the analysis is replaced, and only the conclusion that moved is marked as changed.
A reviewer finds the analysis does not implement the null it claims. The observations stand; the analysis is run again correctly, and each new conclusion names the one it replaces.

17 acts, Researcher and Reviewer.

Expand All @@ -15,24 +15,21 @@ sequenceDiagram
R-->>Researcher: COMP_3
Researcher->>R: conclude x6
R-->>Researcher: COMP_3
Reviewer->>R: recordReview bootstrap is centred on the observed effect it does not impleme…
Researcher->>R: replaceAnalysis sign-flip-permutation
Reviewer->>R: conclude the bootstrap implements the intended null
Researcher->>R: recordAnalysis sign-flip-permutation
R-->>Researcher: COMP_11
Researcher->>R: conclude x6
R-->>Researcher: COMP_11
```

## What moved

`labkit why COMP_11` answers:
`labkit why-supported CLM_13` answers:

**COMP_11** is a revision of COMP_3.
**T beats rewired** — supported, held as exploratory.

- because bootstrap is centred on the observed effect; it does not implement the intended null
- because T beats rewired: p = 0.002 (bootstrap) → p = 0.049 (sign-flip permutation)

- **changed** T beats rewired: p = 0.002 (bootstrap) → p = 0.049 (sign-flip permutation)
- **restated** unchanged: T beats lattice; T beats curr_random; lattice beats curr_random; rewired beats curr_random; T beats static
- **supported by** p = 0.049 (sign-flip permutation)
- **superseded** p = 0.002 (bootstrap) — superseded by "p = 0.049 (sign-flip permutation)"

## The acts, in order

Expand All @@ -47,8 +44,8 @@ sequenceDiagram
| 7 | Researcher | `conclude` | lattice beats curr_random | `COMP_3` |
| 8 | Researcher | `conclude` | rewired beats curr_random | `COMP_3` |
| 9 | Researcher | `conclude` | T beats static | `COMP_3` |
| 10 | Reviewer | `recordReview` | bootstrap is centred on the observed effect; it does not implement the intended… | `REV_10` |
| 11 | Researcher | `replaceAnalysis` | sign-flip-permutation | `COMP_11` |
| 10 | Reviewer | `conclude` | the bootstrap implements the intended null | `COMP_3` |
| 11 | Researcher | `recordAnalysis` | sign-flip-permutation | `COMP_11` |
| 12 | Researcher | `conclude` | T beats lattice | `COMP_11` |
| 13 | Researcher | `conclude` | T beats rewired | `COMP_11` |
| 14 | Researcher | `conclude` | T beats curr_random | `COMP_11` |
Expand Down
32 changes: 0 additions & 32 deletions docs/scenarios/S-11c.md

This file was deleted.

30 changes: 0 additions & 30 deletions docs/scenarios/S-11d.md

This file was deleted.

10 changes: 5 additions & 5 deletions docs/scenarios/S-11e.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ sequenceDiagram
Researcher->>R: recordAnalysis unadjusted comparison
R-->>Researcher: COMP_3
Researcher->>R: conclude the treatment shortens recovery
Reviewer->>R: recordReview unadjusted for baseline severity
Researcher->>R: replaceAnalysis severity-adjusted comparison
Reviewer->>R: note
Researcher->>R: recordAnalysis severity-adjusted comparison
R-->>Researcher: COMP_6
Researcher->>R: conclude the treatment shortens recovery
```
Expand All @@ -27,7 +27,7 @@ sequenceDiagram
**the treatment shortens recovery** — supported, held as exploratory.

- **supported by** one day shorter, adjusted
- **superseded** three days shorter — unadjusted for baseline severity
- **superseded** three days shorter — superseded by "one day shorter, adjusted"

## The acts, in order

Expand All @@ -37,6 +37,6 @@ sequenceDiagram
| 2 | Researcher | `recordObservations` | sixty patients, two arms | `ART_2` |
| 3 | Researcher | `recordAnalysis` | unadjusted comparison | `COMP_3` |
| 4 | Researcher | `conclude` | the treatment shortens recovery | `COMP_3` |
| 5 | Reviewer | `recordReview` | unadjusted for baseline severity | `REV_5` |
| 6 | Researcher | `replaceAnalysis` | severity-adjusted comparison | `COMP_6` |
| 5 | Reviewer | `note` | | `NOTE_5` |
| 6 | Researcher | `recordAnalysis` | severity-adjusted comparison | `COMP_6` |
| 7 | Researcher | `conclude` | the treatment shortens recovery | `COMP_6` |
8 changes: 4 additions & 4 deletions docs/scenarios/S-11g.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ sequenceDiagram
R-->>Researcher: COMP_3
Researcher->>R: conclude x2
R-->>Researcher: COMP_3
Reviewer->>R: recordReview raw-scale aggregation is untrustworthy for the stochastic-contr…
Researcher->>R: keep log-scale re-aggregation
Reviewer->>R: note
Researcher->>R: recordAnalysis log-scale re-aggregation
R-->>Researcher: COMP_7
Researcher->>R: conclude T differs from the current-random control
```
Expand All @@ -30,6 +30,6 @@ sequenceDiagram
| 3 | Researcher | `recordAnalysis` | raw-scale aggregation | `COMP_3` |
| 4 | Researcher | `conclude` | T differs from the current-random control | `COMP_3` |
| 5 | Researcher | `conclude` | T differs from the lattice control | `COMP_3` |
| 6 | Reviewer | `recordReview` | raw-scale aggregation is untrustworthy for the stochastic-control comparisons | `REV_6` |
| 7 | Researcher | `keep` | log-scale re-aggregation | `COMP_7` |
| 6 | Reviewer | `note` | | `NOTE_6` |
| 7 | Researcher | `recordAnalysis` | log-scale re-aggregation | `COMP_7` |
| 8 | Researcher | `conclude` | T differs from the current-random control | `COMP_7` |
Loading
Loading