diff --git a/evidence/511-named-gate-diagnostics/README.md b/evidence/511-named-gate-diagnostics/README.md new file mode 100644 index 000000000..fbe21a911 --- /dev/null +++ b/evidence/511-named-gate-diagnostics/README.md @@ -0,0 +1,60 @@ +# Issue #511: a named gate that fails before its command now says why + +Every transcript here is captured output, redirected to the file by the command +it records. Each begins with the literal invocation and ends with its exit +status. The checkout is Linux x64; the reported incident was macOS arm64 and is +not reproduced by any of these runs. + +Common preamble for every run: `RELAYFLOWD_BIN` is passed explicitly because +`npm run test:prep` exports it inside a subshell that never reaches vitest. The +literal value is +`/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd` — +the kernel target lives outside the tree because `ops/cargo.sh` redirects +`CARGO_TARGET_DIR`. Revisions: head `f308f95`, parent `e21caad`. + +- `head-focused.txt` — the two new files at head: 22 passed, exit 0. +- `mutation-reverted.txt` — mutation verification, red half. + `git checkout e21caad -- src/named-gate-lowering.ts` reverts the only + production file; blob hashes before and after the revert are recorded against + `git rev-parse` of both revisions. Rebuilt, re-run: 13 failed, 9 passed. + The 9 are stream-capture cases that already passed before the change. +- `mutation-restored.txt` — green half. `git checkout HEAD --` restores; + `git hash-object` equals `git rev-parse HEAD:…` + (`8a4ac50391d95c95693eb20cdbc69869a5d02060`) and + `git status --porcelain` is empty, so the restore is byte-for-byte. Rebuilt, + re-run: 22 passed, exit 0. +- `head-package-test.txt` — the required package command, `npm test` from + `packages/sdk`: kernel build, typecheck, build, test typecheck, then vitest. + Complete output including every failure. `Test Files 3 failed | 155 passed | + 1 skipped (159)`, `Tests 30 failed | 2393 passed | 17 skipped (2440)`, + exit 1. The package is not green in this sandbox. +- `head-isolated-three-files.txt` — the three failing files alone at head: + 30 failed, 23 passed, 14 skipped. +- `baseline-in-place-e21caad.txt` — the same three files in the same working + directory with `packages/sdk` reverted to `e21caad` + (`git diff --stat e21caad -- packages/sdk` empty, the two added test files + removed). Identical result: 30 failed, 23 passed, 14 skipped, same cases, + same line numbers. This is the baseline behind the claim that those failures + are pre-existing; the revision is the only variable between it and + `head-isolated-three-files.txt`. +- `restore-identity.txt` — restoring the tree after that baseline. + `git status --porcelain -- packages/sdk` empty and all four changed blobs + hash-identical to `HEAD`. +- `unrelated-failure-cause.txt` — why 22 of those failures happen here: node + resolves `@relayflows/surface` to two different installs, one inside the + checkout and one in an ancestor directory of it + (`/home/daytona/.relayflow-v2-supervisor/durable/node_modules`), so the + handle the workflow creates is absent from the `WeakMap` the test's copy of + `getFlowDefinition` consults (`packages/surface/src/flow.ts:137-143`). A + sandbox layout defect, not a code one. +- `baseline-e21caad.txt` — the same three files at `e21caad` in a separate + `git worktree` under `/tmp` with `node_modules` symlinked. Reported for + completeness and **not** used as the baseline: only 1 failed there, because + `/tmp` has no ancestor `@relayflows` install, which changes the resolution + above. The worktree has since been removed. `baseline-in-place-e21caad.txt` + is the comparison that holds the environment fixed. + +Not run: the Rust suite (no kernel code changed) and any Cloud runner-log +publication, whose producer does not exist in this repository. Nothing here +demonstrates the `flows logs ` acceptance item; `summary.md` records it +as unmet. diff --git a/evidence/511-named-gate-diagnostics/baseline-e21caad.txt b/evidence/511-named-gate-diagnostics/baseline-e21caad.txt new file mode 100644 index 000000000..6e1574a27 --- /dev/null +++ b/evidence/511-named-gate-diagnostics/baseline-e21caad.txt @@ -0,0 +1,107 @@ +# Baseline run at the parent revision e21caad, in a detached git worktree. +$ git -C /tmp/flows-baseline-e21caad rev-parse HEAD +e21caad1e4020d9e2675d2840301cb7eeb6f1aac +$ git -C /tmp/flows-baseline-e21caad status --porcelain +?? packages/sdk/node_modules +# node_modules is a symlink to the main checkout (npm install is not rerun): +$ ls -l /tmp/flows-baseline-e21caad/packages/sdk/node_modules +lrwxrwxrwx 1 daytona daytona 83 Sep 20 17:41 /tmp/flows-baseline-e21caad/packages/sdk/node_modules -> /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/node_modules + +$ RELAYFLOWD_BIN=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd npm run build && npx vitest run tests/live-kernel.test.ts tests/stuck-run-triage.test.ts tests/authored-node-runtime.test.ts + +> @relayflows/sdk@2.0.22 build +> tsc && node scripts/make-cli-executable.mjs + + + RUN v2.1.9 /tmp/flows-baseline-e21caad/packages/sdk + + ❯ tests/authored-node-runtime.test.ts (14 tests | 14 skipped) 19ms +stdout | tests/live-kernel.test.ts +LIVE_KERNEL relayflowd=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd +LIVE_KERNEL flows=/tmp/flows-baseline-e21caad/packages/sdk/dist/cli.js + + ✓ tests/stuck-run-triage.test.ts (22 tests) 3143ms + ✓ stuck-run-triage shell text > collects tails with no GNU timeout on PATH, as on a stock macOS 3084ms +stdout | tests/live-kernel.test.ts > built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI +LIVE_ANALYZER ready: claude -p --model claude-haiku-4-5-20251001 round-trip OK + +stdout | tests/live-kernel.test.ts > surface resume after a real daemon kill > resumes a three-step run with each successful completion exactly once +LIVE_KERNEL kill -9 pid=80459 run=01M2ZYKSAN5HW6C58HHTTVPSA8 while step=two state=Running + + ❯ tests/live-kernel.test.ts (31 tests | 1 failed) 56356ms + ✓ built flows CLI against live relayflowd > twenty-six-step reuses 25 durable completions after editing the failed final step 1953ms + ✓ built flows CLI against live relayflowd > runs rung (a), parks rung (b), and keeps JSON report-shaped 2298ms + ✓ built flows CLI against live relayflowd > allows a deterministic run to exceed the bounded request timeout 32442ms + ✓ built flows CLI against live relayflowd > follows a live worker dispatch through flows run 567ms + ✓ built flows CLI against live relayflowd > runs an agent CLI end to end through the SDK worker 472ms + ✓ built flows CLI against live relayflowd > f.agent lowers to a real agent step and dispatches through a live worker 576ms + ✓ built flows CLI against live relayflowd > can always get a parked run to a late-attaching worker 5577ms + ✓ built flows CLI against live relayflowd > reports a real manual-recovery NeedsHuman state as parked 413ms + ✓ built flows CLI against live relayflowd > runs hn-monitor analyze-story end-to-end via a stub agent CLI (gate 2 clause 2 demo) 418ms + ✓ built flows CLI against live relayflowd > hn-monitor analyze-story FAILS verification when the CLI omits required schema fields 440ms + ✓ built flows CLI against live relayflowd > agent step preserves the CliResult wrapper as output when the CLI emits non-JSON text 460ms + ✓ built flows CLI against live relayflowd > AgentWorker exposes wake_context to the CLI via RELAYFLOW_WAKE_CONTEXT env var (real analyzer prerequisite) 527ms + ✓ built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_WAKE_CONTEXT UNSET when the run has no wake_context (undefined-vs-null pin) 534ms + ✓ built flows CLI against live relayflowd > AgentWorker passes a declared model to an identified wrapper as RELAYFLOW_MODEL 634ms + ✓ built flows CLI against live relayflowd > AgentWorker refuses a nonconforming journal-submitted wrapper before exposing RELAYFLOW_MODEL 470ms + ✓ built flows CLI against live relayflowd > AgentWorker executes the raw claude adapter with its real model flag 432ms + ✓ built flows CLI against live relayflowd > AgentWorker executes the raw codex adapter with its real model flag 479ms + ✓ built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_MODEL UNSET when the step declares no model 541ms + × built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI 3805ms + → expected { …(3) } to match object { Object (gate, verdict) } +(1 matching property omitted from actual) + ✓ built flows CLI against live relayflowd > preflights before journaling and names an unreachable socket 720ms + ✓ built flows CLI against live relayflowd > starts exactly one daemon when two runs race for one empty data dir 431ms + ✓ surface resume after a real daemon kill > resumes a three-step run with each successful completion exactly once 807ms + ✓ a relayflow can be scheduled: tick source against live relayflowd > a tick spawns a real run whose step reports the SCHEDULED instant 627ms + +⎯⎯⎯⎯⎯⎯ Failed Suites 1 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/authored-node-runtime.test.ts [ tests/authored-node-runtime.test.ts ] +AssertionError: expected '1.3.6' to be '1.4.0' // Object.is equality + +Expected: "1.4.0" +Received: "1.3.6" + + ❯ tests/authored-node-runtime.test.ts:18:77 + 16| + 17| beforeAll(() => { + 18| expect(spawnSync(bun, ['--version'], { encoding: 'utf8' }).stdout.tr… + | ^ + 19| expect(existsSync(daemon), 'build the current kernel or set RELAYFLO… + 20| stage = mkdtempSync(join(tmpdir(), 'authored-standalone-build-')); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/2]⎯ + +⎯⎯⎯⎯⎯⎯⎯ Failed Tests 1 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI +AssertionError: expected { …(3) } to match object { Object (gate, verdict) } +(1 matching property omitted from actual) + +- Expected ++ Received + + Object { +- "gate": "json_schema", +- "verdict": "pass", ++ "gate": "execution", ++ "verdict": "fail", + } + + ❯ tests/live-kernel.test.ts:1317:49 + 1315| // the promoted output, not the test re-deriving it: the spec is + 1316| // the unmodified canonical one, so this record is the gate. + 1317| expect(stepCompleted!.payload.verification).toMatchObject({ + | ^ + 1318| gate: 'json_schema', + 1319| verdict: 'pass', + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/2]⎯ + + Test Files 2 failed | 1 passed (3) + Tests 1 failed | 52 passed | 14 skipped (67) + Start at 17:41:55 + Duration 57.63s (transform 1.01s, setup 0ms, collect 2.40s, tests 59.52s, environment 0ms, prepare 114ms) + +EXIT=1 diff --git a/evidence/511-named-gate-diagnostics/baseline-in-place-e21caad.txt b/evidence/511-named-gate-diagnostics/baseline-in-place-e21caad.txt new file mode 100644 index 000000000..69d2c02e3 --- /dev/null +++ b/evidence/511-named-gate-diagnostics/baseline-in-place-e21caad.txt @@ -0,0 +1,560 @@ +# In-place baseline: same working directory and environment as the HEAD run, +# with packages/sdk reverted to the parent revision e21caad. This isolates the +# revision as the only variable. +$ git checkout e21caad -- packages/sdk +$ rm packages/sdk/tests/named-gate-diagnostics.test.ts packages/sdk/tests/named-gate-journal.test.ts # files this PR adds +$ git diff --stat e21caad -- packages/sdk # empty = package content equals the parent revision +$ git hash-object packages/sdk/src/named-gate-lowering.ts; git rev-parse e21caad:packages/sdk/src/named-gate-lowering.ts +479136c28de4f97c5434800577999bbbb1eca59c +479136c28de4f97c5434800577999bbbb1eca59c + +$ cd packages/sdk && RELAYFLOWD_BIN=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd npm run build && npx vitest run tests/live-kernel.test.ts tests/stuck-run-triage.test.ts tests/authored-node-runtime.test.ts + +> @relayflows/sdk@2.0.22 build +> tsc && node scripts/make-cli-executable.mjs + + + RUN v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk + + ❯ tests/stuck-run-triage.test.ts (22 tests | 22 failed) 77ms + × stuck-run-triage input validation > refuses an 8-character run-id prefix: Cloud has no prefix lookup 4ms + → expected [Function] to throw error matching /not full Cloud run ids: c649fe14/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses the whole batch when any id is invalid, rather than dropping it 1ms + → expected [Function] to throw error matching /not full Cloud run ids: nope!/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses an empty batch 0ms + → expected [Function] to throw error matching /needs runIds/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses a batch too large for the edge step lease 0ms + → expected [Function] to throw error matching /exceeds the 8 that fit/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > accepts eight ids — the incident batch is inside the bound 3ms + → promise rejected "TypeError: expected an @relayflows/surfac…" instead of resolving + × stuck-run-triage apiUrl > refuses to send the Cloud bearer token to an unapproved origin 0ms + → expected [Function] to throw error matching /refusing to send the Cloud bearer to…/\ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage apiUrl > refuses a non-URL apiUrl 0ms + → expected [Function] to throw error matching /is not a URL/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage apiUrl > allows an approved origin and uses it in the curl 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage apiUrl > defaults to production Cloud 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage apiUrl > never publishes a run record the fetch did not produce 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > names the Worker on every wrangler invocation 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > accepts caller-supplied Workers and rejects option-shaped ones 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > falls back when GNU timeout is absent, as it is on macOS 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > runs the tails concurrently so wall time does not scale with the batch 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > records wrangler's own exit status rather than head's 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage shell text > parses under both sh and bash 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage shell text > collects tails with no GNU timeout on PATH, as on a stock macOS 61ms + → expected an @relayflows/surface flow handle + × stuck-run-triage agents > declares read-only permissions on every agent 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage agents > tells the forensics agents their evidence is untrusted 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage fan-out > refuses a duplicate run id: two tails would share one evidence file 1ms + → expected [Function] to throw error matching /duplicate runIds: c649fe14-0c2e-4e51-…/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage fan-out > refuses a duplicate Worker name for the same reason 0ms + → expected [Function] to throw error matching /duplicate workers: w-one/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage fan-out > bounds ids x workers, not just ids 0ms + → expected [Function] to throw error matching /24 concurrent tails, over the 16/ but got 'expected an @relayflows/surface flow …' + ❯ tests/authored-node-runtime.test.ts (14 tests | 14 skipped) 12ms +stdout | tests/live-kernel.test.ts +LIVE_KERNEL relayflowd=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd +LIVE_KERNEL flows=/home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/dist/cli.js + +stdout | tests/live-kernel.test.ts > surface resume after a real daemon kill > resumes a three-step run with each successful completion exactly once +LIVE_KERNEL kill -9 pid=83147 run=01M2ZYXA39PCN57YRZ2FNSDGJ8 while step=two state=Running + + ❯ tests/live-kernel.test.ts (31 tests | 8 failed) 52176ms + ✓ built flows CLI against live relayflowd > twenty-six-step reuses 25 durable completions after editing the failed final step 1987ms + ✓ built flows CLI against live relayflowd > runs rung (a), parks rung (b), and keeps JSON report-shaped 2302ms + ✓ built flows CLI against live relayflowd > allows a deterministic run to exceed the bounded request timeout 32424ms + ✓ built flows CLI against live relayflowd > follows a live worker dispatch through flows run 547ms + ✓ built flows CLI against live relayflowd > runs an agent CLI end to end through the SDK worker 367ms + ✓ built flows CLI against live relayflowd > f.agent lowers to a real agent step and dispatches through a live worker 545ms + ✓ built flows CLI against live relayflowd > can always get a parked run to a late-attaching worker 5577ms + ✓ built flows CLI against live relayflowd > reports a real manual-recovery NeedsHuman state as parked 418ms + × built flows CLI against live relayflowd > runs hn-monitor analyze-story end-to-end via a stub agent CLI (gate 2 clause 2 demo) 568ms + → expected { …(12) } to match object { output: { …(3) }, …(1) } +(22 matching properties omitted from actual) + × built flows CLI against live relayflowd > hn-monitor analyze-story FAILS verification when the CLI omits required schema fields 548ms + → expected { …(12) } to match object { …(3) } +(21 matching properties omitted from actual) + × built flows CLI against live relayflowd > agent step preserves the CliResult wrapper as output when the CLI emits non-JSON text 519ms + → expected null not to be null + × built flows CLI against live relayflowd > AgentWorker exposes wake_context to the CLI via RELAYFLOW_WAKE_CONTEXT env var (real analyzer prerequisite) 493ms + → Cannot read properties of null (reading 'story_title') + × built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_WAKE_CONTEXT UNSET when the run has no wake_context (undefined-vs-null pin) 503ms + → Cannot read properties of null (reading 'env_present') + ✓ built flows CLI against live relayflowd > AgentWorker passes a declared model to an identified wrapper as RELAYFLOW_MODEL 565ms + ✓ built flows CLI against live relayflowd > AgentWorker refuses a nonconforming journal-submitted wrapper before exposing RELAYFLOW_MODEL 415ms + ✓ built flows CLI against live relayflowd > AgentWorker executes the raw claude adapter with its real model flag 371ms + ✓ built flows CLI against live relayflowd > AgentWorker executes the raw codex adapter with its real model flag 407ms + × built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_MODEL UNSET when the step declares no model 439ms + → Cannot read properties of null (reading 'story_title') + × built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI 37ms + → LIVE_ANALYZER_UNAVAILABLE: "/home/daytona/.relayflow-v2-supervisor/durable/repository/testdata/preflight/analyze-story-claude-cli" does not identify as relayflows-agent-cli-v1 — failing because gate-2 acceptance requires the real analyzer to execute. Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is not gate evidence. + ✓ built flows CLI against live relayflowd > preflights before journaling and names an unreachable socket 725ms + ✓ built flows CLI against live relayflowd > starts exactly one daemon when two runs race for one empty data dir 449ms + ✓ surface resume after a real daemon kill > resumes a three-step run with each successful completion exactly once 813ms + × a relayflow can be scheduled: tick source against live relayflowd > a tick spawns a real run whose step reports the SCHEDULED instant 436ms + → expected null to deeply equal { schedule_id: 'heartbeat-1m', …(3) } + +⎯⎯⎯⎯⎯⎯ Failed Suites 1 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/authored-node-runtime.test.ts [ tests/authored-node-runtime.test.ts ] +AssertionError: expected '1.3.6' to be '1.4.0' // Object.is equality + +Expected: "1.4.0" +Received: "1.3.6" + + ❯ tests/authored-node-runtime.test.ts:18:77 + 16| + 17| beforeAll(() => { + 18| expect(spawnSync(bun, ['--version'], { encoding: 'utf8' }).stdout.tr… + | ^ + 19| expect(existsSync(daemon), 'build the current kernel or set RELAYFLO… + 20| stage = mkdtempSync(join(tmpdir(), 'authored-standalone-build-')); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/31]⎯ + +⎯⎯⎯⎯⎯⎯ Failed Tests 30 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > runs hn-monitor analyze-story end-to-end via a stub agent CLI (gate 2 clause 2 demo) +AssertionError: expected { …(12) } to match object { output: { …(3) }, …(1) } +(22 matching properties omitted from actual) + +- Expected ++ Received + + Object { +- "output": Object { +- "reasoning": "stub agent runtime — deterministic output for gate-2 clause-2 demo", +- "relevance_score": 5, +- "story_title": "stub", +- }, ++ "output": null, + "verification": Object { +- "gate": "json_schema", +- "verdict": "pass", ++ "gate": "execution", ++ "verdict": "fail", + }, + } + + ❯ tests/live-kernel.test.ts:657:36 + 655| && (entry as { step_id?: string }).step_id === 'analyze-story', + 656| ) as { payload: { output: unknown; verification: unknown } } | und… + 657| expect(stepCompleted?.payload).toMatchObject({ + | ^ + 658| output: { + 659| story_title: 'stub', + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > hn-monitor analyze-story FAILS verification when the CLI omits required schema fields +AssertionError: expected { …(12) } to match object { …(3) } +(21 matching properties omitted from actual) + +- Expected ++ Received + + Object { +- "completionReason": "retries_exhausted", ++ "completionReason": "worker_error", + "output": null, + "verification": Object { +- "gate": "json_schema", ++ "gate": "execution", + "verdict": "fail", + }, + } + + ❯ tests/live-kernel.test.ts:752:36 + 750| // its verification record names the json_schema rejection. The re… + 751| // parsed value is nulled before the completion is persisted. + 752| expect(stepCompleted?.payload).toMatchObject({ + | ^ + 753| completionReason: 'retries_exhausted', + 754| output: null, + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[3/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > agent step preserves the CliResult wrapper as output when the CLI emits non-JSON text +AssertionError: expected null not to be null + ❯ tests/live-kernel.test.ts:823:24 + 821| // here (parseJsonOutput returned null on non-JSON stdout) and + 822| // these assertions would all fail. + 823| expect(output).not.toBeNull(); + | ^ + 824| expect(output.exit_code).toBe(0); + 825| expect(output.stdout_tail).toContain('looked at the story'); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[4/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker exposes wake_context to the CLI via RELAYFLOW_WAKE_CONTEXT env var (real analyzer prerequisite) +TypeError: Cannot read properties of null (reading 'story_title') + ❯ tests/live-kernel.test.ts:891:42 + 889| ) as { payload: { output: { story_title: string; reasoning: string… + 890| expect(stepCompleted).toBeDefined(); + 891| expect(stepCompleted!.payload.output.story_title).toBe(`echoed:${s… + | ^ + 892| expect(stepCompleted!.payload.output.reasoning).toContain(String(s… + 893| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[5/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_WAKE_CONTEXT UNSET when the run has no wake_context (undefined-vs-null pin) +TypeError: Cannot read properties of null (reading 'env_present') + ❯ tests/live-kernel.test.ts:958:38 + 956| ) as { payload: { output: { env_present: boolean } } } | undefined; + 957| expect(completed).toBeDefined(); + 958| expect(completed!.payload.output.env_present).toBe(false); + | ^ + 959| + 960| delete process.env.RELAYFLOW_WAKE_CONTEXT; + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[6/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_MODEL UNSET when the step declares no model +TypeError: Cannot read properties of null (reading 'story_title') + ❯ tests/live-kernel.test.ts:1194:38 + 1192| expect(completed).toBeDefined(); + 1193| // UNSET, not EMPTY and not the leaked parent value. + 1194| expect(completed!.payload.output.story_title).toBe('model:UNSET'); + | ^ + 1195| + 1196| delete process.env.RELAYFLOW_MODEL; + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[7/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI +Error: LIVE_ANALYZER_UNAVAILABLE: "/home/daytona/.relayflow-v2-supervisor/durable/repository/testdata/preflight/analyze-story-claude-cli" does not identify as relayflows-agent-cli-v1 — failing because gate-2 acceptance requires the real analyzer to execute. Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is not gate evidence. + ❯ tests/live-kernel.test.ts:1223:15 + 1221| const notice = `LIVE_ANALYZER_UNAVAILABLE: ${readiness.detail}`; + 1222| if (process.env['RELAYFLOWS_ALLOW_ANALYZER_SKIP'] !== '1') { + 1223| throw new Error( + | ^ + 1224| `${notice} — failing because gate-2 acceptance requires the … + 1225| + 'Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is … + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[8/31]⎯ + + FAIL tests/live-kernel.test.ts > a relayflow can be scheduled: tick source against live relayflowd > a tick spawns a real run whose step reports the SCHEDULED instant +AssertionError: expected null to deeply equal { schedule_id: 'heartbeat-1m', …(3) } + +- Expected: +Object { + "lag_ms": 43000, + "schedule_id": "heartbeat-1m", + "scheduled_for_ms": 1764000000000, + "slot": 29400000, +} + ++ Received: +null + + ❯ tests/live-kernel.test.ts:1665:39 + 1663| // The bound: the run reports the grid instant and its own lag, so… + 1664| // backfilled run can tell it is running for a slot from the past. + 1665| expect(completed!.payload.output).toEqual({ + | ^ + 1666| schedule_id: 'heartbeat-1m', + 1667| slot: 29_400_000, + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[9/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses an 8-character run-id prefix: Cloud has no prefix lookup +AssertionError: expected [Function] to throw error matching /not full Cloud run ids: c649fe14/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/not full Cloud run ids: c649fe14/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[10/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses the whole batch when any id is invalid, rather than dropping it +AssertionError: expected [Function] to throw error matching /not full Cloud run ids: nope!/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/not full Cloud run ids: nope!/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[11/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses an empty batch +AssertionError: expected [Function] to throw error matching /needs runIds/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/needs runIds/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[12/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses a batch too large for the edge step lease +AssertionError: expected [Function] to throw error matching /exceeds the 8 that fit/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/exceeds the 8 that fit/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[13/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > accepts eight ids — the incident batch is inside the bound +AssertionError: promise rejected "TypeError: expected an @relayflows/surfac…" instead of resolving + ❯ tests/stuck-run-triage.test.ts:62:40 + 60| it('accepts eight ids — the incident batch is inside the bound', asy… + 61| const ids = Array.from({ length: 8 }, (_, i) => `${ID_A.slice(0, -… + 62| await expect(drive({ runIds: ids })).resolves.toBeDefined(); + | ^ + 63| }); + 64| }); + +Caused by: TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + ❯ tests/stuck-run-triage.test.ts:62:18 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[14/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > refuses to send the Cloud bearer token to an unapproved origin +AssertionError: expected [Function] to throw error matching /refusing to send the Cloud bearer to…/\ but got 'expected an @relayflows/surface flow …' + +- Expected: +/refusing to send the Cloud bearer token to https:\/\/evil\.example/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[15/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > refuses a non-URL apiUrl +AssertionError: expected [Function] to throw error matching /is not a URL/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/is not a URL/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[16/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > allows an approved origin and uses it in the curl +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:77:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[17/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > defaults to production Cloud +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:82:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[18/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > never publishes a run record the fetch did not produce +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:89:34 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[19/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > names the Worker on every wrangler invocation +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:98:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[20/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > accepts caller-supplied Workers and rejects option-shaped ones +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:107:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[21/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > falls back when GNU timeout is absent, as it is on macOS +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:115:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[22/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > runs the tails concurrently so wall time does not scale with the batch +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:123:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[23/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > records wrangler's own exit status rather than head's +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:129:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[24/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage shell text > parses under both sh and bash +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:137:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[25/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage shell text > collects tails with no GNU timeout on PATH, as on a stock macOS +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:157:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[26/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage agents > declares read-only permissions on every agent +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:176:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[27/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage agents > tells the forensics agents their evidence is untrusted +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:182:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[28/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > refuses a duplicate run id: two tails would share one evidence file +AssertionError: expected [Function] to throw error matching /duplicate runIds: c649fe14-0c2e-4e51-…/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/duplicate runIds: c649fe14-0c2e-4e51-9a6a-4f0d1b0f77aa/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[29/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > refuses a duplicate Worker name for the same reason +AssertionError: expected [Function] to throw error matching /duplicate workers: w-one/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/duplicate workers: w-one/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[30/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > bounds ids x workers, not just ids +AssertionError: expected [Function] to throw error matching /24 concurrent tails, over the 16/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/24 concurrent tails, over the 16/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[31/31]⎯ + + Test Files 3 failed (3) + Tests 30 failed | 23 passed | 14 skipped (67) + Start at 17:47:11 + Duration 53.45s (transform 943ms, setup 0ms, collect 2.43s, tests 52.26s, environment 0ms, prepare 149ms) + +EXIT=1 diff --git a/evidence/511-named-gate-diagnostics/head-focused.txt b/evidence/511-named-gate-diagnostics/head-focused.txt new file mode 100644 index 000000000..841114778 --- /dev/null +++ b/evidence/511-named-gate-diagnostics/head-focused.txt @@ -0,0 +1,25 @@ +$ git rev-parse HEAD +f308f95683c46eaddcf5c22eefd35aa04d2d1500 +$ git status --porcelain -- src tests tsconfig.tests.json +$ git hash-object src/named-gate-lowering.ts +8a4ac50391d95c95693eb20cdbc69869a5d02060 +$ export RELAYFLOWD_BIN=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd +$ npm run build && npx vitest run tests/named-gate-diagnostics.test.ts tests/named-gate-journal.test.ts + +> @relayflows/sdk@2.0.22 build +> tsc && node scripts/make-cli-executable.mjs + + + RUN v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk + + ✓ tests/named-gate-diagnostics.test.ts (17 tests) 535ms + ✓ tests/named-gate-journal.test.ts (5 tests) 1664ms + ✓ a lowered subprocess_gate journals the gate command's streams > persists what the gate printed before a timeout killed it 816ms + ✓ a gate on an agent step > journals the command's streams and surfaces them in the authored failure 582ms + + Test Files 2 passed (2) + Tests 22 passed (22) + Start at 17:40:34 + Duration 2.88s (transform 692ms, setup 0ms, collect 1.46s, tests 2.20s, environment 0ms, prepare 75ms) + +EXIT=0 diff --git a/evidence/511-named-gate-diagnostics/head-isolated-three-files.txt b/evidence/511-named-gate-diagnostics/head-isolated-three-files.txt new file mode 100644 index 000000000..57a10f8fd --- /dev/null +++ b/evidence/511-named-gate-diagnostics/head-isolated-three-files.txt @@ -0,0 +1,555 @@ +# Same three files at HEAD (f308f95), run in isolation, for a like-for-like comparison with baseline-e21caad.txt. +$ git rev-parse HEAD +f308f95683c46eaddcf5c22eefd35aa04d2d1500 +$ git status --porcelain -- src tests tsconfig.tests.json + +$ RELAYFLOWD_BIN=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd npm run build && npx vitest run tests/live-kernel.test.ts tests/stuck-run-triage.test.ts tests/authored-node-runtime.test.ts + +> @relayflows/sdk@2.0.22 build +> tsc && node scripts/make-cli-executable.mjs + + + RUN v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk + + ❯ tests/stuck-run-triage.test.ts (22 tests | 22 failed) 99ms + × stuck-run-triage input validation > refuses an 8-character run-id prefix: Cloud has no prefix lookup 5ms + → expected [Function] to throw error matching /not full Cloud run ids: c649fe14/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses the whole batch when any id is invalid, rather than dropping it 1ms + → expected [Function] to throw error matching /not full Cloud run ids: nope!/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses an empty batch 0ms + → expected [Function] to throw error matching /needs runIds/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses a batch too large for the edge step lease 0ms + → expected [Function] to throw error matching /exceeds the 8 that fit/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > accepts eight ids — the incident batch is inside the bound 20ms + → promise rejected "TypeError: expected an @relayflows/surfac…" instead of resolving + × stuck-run-triage apiUrl > refuses to send the Cloud bearer token to an unapproved origin 1ms + → expected [Function] to throw error matching /refusing to send the Cloud bearer to…/\ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage apiUrl > refuses a non-URL apiUrl 1ms + → expected [Function] to throw error matching /is not a URL/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage apiUrl > allows an approved origin and uses it in the curl 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage apiUrl > defaults to production Cloud 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage apiUrl > never publishes a run record the fetch did not produce 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > names the Worker on every wrangler invocation 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > accepts caller-supplied Workers and rejects option-shaped ones 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > falls back when GNU timeout is absent, as it is on macOS 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > runs the tails concurrently so wall time does not scale with the batch 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > records wrangler's own exit status rather than head's 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage shell text > parses under both sh and bash 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage shell text > collects tails with no GNU timeout on PATH, as on a stock macOS 65ms + → expected an @relayflows/surface flow handle + × stuck-run-triage agents > declares read-only permissions on every agent 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage agents > tells the forensics agents their evidence is untrusted 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage fan-out > refuses a duplicate run id: two tails would share one evidence file 1ms + → expected [Function] to throw error matching /duplicate runIds: c649fe14-0c2e-4e51-…/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage fan-out > refuses a duplicate Worker name for the same reason 0ms + → expected [Function] to throw error matching /duplicate workers: w-one/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage fan-out > bounds ids x workers, not just ids 1ms + → expected [Function] to throw error matching /24 concurrent tails, over the 16/ but got 'expected an @relayflows/surface flow …' + ❯ tests/authored-node-runtime.test.ts (14 tests | 14 skipped) 21ms +stdout | tests/live-kernel.test.ts +LIVE_KERNEL relayflowd=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd +LIVE_KERNEL flows=/home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/dist/cli.js + +stdout | tests/live-kernel.test.ts > surface resume after a real daemon kill > resumes a three-step run with each successful completion exactly once +LIVE_KERNEL kill -9 pid=81494 run=01M2ZYPBSJ865AJXCAN8WSB46S while step=two state=Running + + ❯ tests/live-kernel.test.ts (31 tests | 8 failed) 53093ms + ✓ built flows CLI against live relayflowd > twenty-six-step reuses 25 durable completions after editing the failed final step 2014ms + ✓ built flows CLI against live relayflowd > runs rung (a), parks rung (b), and keeps JSON report-shaped 2275ms + ✓ built flows CLI against live relayflowd > allows a deterministic run to exceed the bounded request timeout 32429ms + ✓ built flows CLI against live relayflowd > follows a live worker dispatch through flows run 533ms + ✓ built flows CLI against live relayflowd > runs an agent CLI end to end through the SDK worker 469ms + ✓ built flows CLI against live relayflowd > f.agent lowers to a real agent step and dispatches through a live worker 749ms + ✓ built flows CLI against live relayflowd > can always get a parked run to a late-attaching worker 5577ms + ✓ built flows CLI against live relayflowd > reports a real manual-recovery NeedsHuman state as parked 464ms + × built flows CLI against live relayflowd > runs hn-monitor analyze-story end-to-end via a stub agent CLI (gate 2 clause 2 demo) 513ms + → expected { …(12) } to match object { output: { …(3) }, …(1) } +(22 matching properties omitted from actual) + × built flows CLI against live relayflowd > hn-monitor analyze-story FAILS verification when the CLI omits required schema fields 614ms + → expected { …(12) } to match object { …(3) } +(21 matching properties omitted from actual) + × built flows CLI against live relayflowd > agent step preserves the CliResult wrapper as output when the CLI emits non-JSON text 573ms + → expected null not to be null + × built flows CLI against live relayflowd > AgentWorker exposes wake_context to the CLI via RELAYFLOW_WAKE_CONTEXT env var (real analyzer prerequisite) 607ms + → Cannot read properties of null (reading 'story_title') + × built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_WAKE_CONTEXT UNSET when the run has no wake_context (undefined-vs-null pin) 499ms + → Cannot read properties of null (reading 'env_present') + ✓ built flows CLI against live relayflowd > AgentWorker passes a declared model to an identified wrapper as RELAYFLOW_MODEL 568ms + ✓ built flows CLI against live relayflowd > AgentWorker refuses a nonconforming journal-submitted wrapper before exposing RELAYFLOW_MODEL 557ms + ✓ built flows CLI against live relayflowd > AgentWorker executes the raw claude adapter with its real model flag 400ms + ✓ built flows CLI against live relayflowd > AgentWorker executes the raw codex adapter with its real model flag 488ms + × built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_MODEL UNSET when the step declares no model 446ms + → Cannot read properties of null (reading 'story_title') + × built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI 33ms + → LIVE_ANALYZER_UNAVAILABLE: "/home/daytona/.relayflow-v2-supervisor/durable/repository/testdata/preflight/analyze-story-claude-cli" does not identify as relayflows-agent-cli-v1 — failing because gate-2 acceptance requires the real analyzer to execute. Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is not gate evidence. + ✓ built flows CLI against live relayflowd > preflights before journaling and names an unreachable socket 731ms + ✓ built flows CLI against live relayflowd > starts exactly one daemon when two runs race for one empty data dir 439ms + ✓ surface resume after a real daemon kill > resumes a three-step run with each successful completion exactly once 817ms + × a relayflow can be scheduled: tick source against live relayflowd > a tick spawns a real run whose step reports the SCHEDULED instant 514ms + → expected null to deeply equal { schedule_id: 'heartbeat-1m', …(3) } + +⎯⎯⎯⎯⎯⎯ Failed Suites 1 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/authored-node-runtime.test.ts [ tests/authored-node-runtime.test.ts ] +AssertionError: expected '1.3.6' to be '1.4.0' // Object.is equality + +Expected: "1.4.0" +Received: "1.3.6" + + ❯ tests/authored-node-runtime.test.ts:18:77 + 16| + 17| beforeAll(() => { + 18| expect(spawnSync(bun, ['--version'], { encoding: 'utf8' }).stdout.tr… + | ^ + 19| expect(existsSync(daemon), 'build the current kernel or set RELAYFLO… + 20| stage = mkdtempSync(join(tmpdir(), 'authored-standalone-build-')); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/31]⎯ + +⎯⎯⎯⎯⎯⎯ Failed Tests 30 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > runs hn-monitor analyze-story end-to-end via a stub agent CLI (gate 2 clause 2 demo) +AssertionError: expected { …(12) } to match object { output: { …(3) }, …(1) } +(22 matching properties omitted from actual) + +- Expected ++ Received + + Object { +- "output": Object { +- "reasoning": "stub agent runtime — deterministic output for gate-2 clause-2 demo", +- "relevance_score": 5, +- "story_title": "stub", +- }, ++ "output": null, + "verification": Object { +- "gate": "json_schema", +- "verdict": "pass", ++ "gate": "execution", ++ "verdict": "fail", + }, + } + + ❯ tests/live-kernel.test.ts:657:36 + 655| && (entry as { step_id?: string }).step_id === 'analyze-story', + 656| ) as { payload: { output: unknown; verification: unknown } } | und… + 657| expect(stepCompleted?.payload).toMatchObject({ + | ^ + 658| output: { + 659| story_title: 'stub', + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > hn-monitor analyze-story FAILS verification when the CLI omits required schema fields +AssertionError: expected { …(12) } to match object { …(3) } +(21 matching properties omitted from actual) + +- Expected ++ Received + + Object { +- "completionReason": "retries_exhausted", ++ "completionReason": "worker_error", + "output": null, + "verification": Object { +- "gate": "json_schema", ++ "gate": "execution", + "verdict": "fail", + }, + } + + ❯ tests/live-kernel.test.ts:752:36 + 750| // its verification record names the json_schema rejection. The re… + 751| // parsed value is nulled before the completion is persisted. + 752| expect(stepCompleted?.payload).toMatchObject({ + | ^ + 753| completionReason: 'retries_exhausted', + 754| output: null, + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[3/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > agent step preserves the CliResult wrapper as output when the CLI emits non-JSON text +AssertionError: expected null not to be null + ❯ tests/live-kernel.test.ts:823:24 + 821| // here (parseJsonOutput returned null on non-JSON stdout) and + 822| // these assertions would all fail. + 823| expect(output).not.toBeNull(); + | ^ + 824| expect(output.exit_code).toBe(0); + 825| expect(output.stdout_tail).toContain('looked at the story'); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[4/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker exposes wake_context to the CLI via RELAYFLOW_WAKE_CONTEXT env var (real analyzer prerequisite) +TypeError: Cannot read properties of null (reading 'story_title') + ❯ tests/live-kernel.test.ts:891:42 + 889| ) as { payload: { output: { story_title: string; reasoning: string… + 890| expect(stepCompleted).toBeDefined(); + 891| expect(stepCompleted!.payload.output.story_title).toBe(`echoed:${s… + | ^ + 892| expect(stepCompleted!.payload.output.reasoning).toContain(String(s… + 893| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[5/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_WAKE_CONTEXT UNSET when the run has no wake_context (undefined-vs-null pin) +TypeError: Cannot read properties of null (reading 'env_present') + ❯ tests/live-kernel.test.ts:958:38 + 956| ) as { payload: { output: { env_present: boolean } } } | undefined; + 957| expect(completed).toBeDefined(); + 958| expect(completed!.payload.output.env_present).toBe(false); + | ^ + 959| + 960| delete process.env.RELAYFLOW_WAKE_CONTEXT; + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[6/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_MODEL UNSET when the step declares no model +TypeError: Cannot read properties of null (reading 'story_title') + ❯ tests/live-kernel.test.ts:1194:38 + 1192| expect(completed).toBeDefined(); + 1193| // UNSET, not EMPTY and not the leaked parent value. + 1194| expect(completed!.payload.output.story_title).toBe('model:UNSET'); + | ^ + 1195| + 1196| delete process.env.RELAYFLOW_MODEL; + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[7/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI +Error: LIVE_ANALYZER_UNAVAILABLE: "/home/daytona/.relayflow-v2-supervisor/durable/repository/testdata/preflight/analyze-story-claude-cli" does not identify as relayflows-agent-cli-v1 — failing because gate-2 acceptance requires the real analyzer to execute. Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is not gate evidence. + ❯ tests/live-kernel.test.ts:1223:15 + 1221| const notice = `LIVE_ANALYZER_UNAVAILABLE: ${readiness.detail}`; + 1222| if (process.env['RELAYFLOWS_ALLOW_ANALYZER_SKIP'] !== '1') { + 1223| throw new Error( + | ^ + 1224| `${notice} — failing because gate-2 acceptance requires the … + 1225| + 'Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is … + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[8/31]⎯ + + FAIL tests/live-kernel.test.ts > a relayflow can be scheduled: tick source against live relayflowd > a tick spawns a real run whose step reports the SCHEDULED instant +AssertionError: expected null to deeply equal { schedule_id: 'heartbeat-1m', …(3) } + +- Expected: +Object { + "lag_ms": 43000, + "schedule_id": "heartbeat-1m", + "scheduled_for_ms": 1764000000000, + "slot": 29400000, +} + ++ Received: +null + + ❯ tests/live-kernel.test.ts:1665:39 + 1663| // The bound: the run reports the grid instant and its own lag, so… + 1664| // backfilled run can tell it is running for a slot from the past. + 1665| expect(completed!.payload.output).toEqual({ + | ^ + 1666| schedule_id: 'heartbeat-1m', + 1667| slot: 29_400_000, + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[9/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses an 8-character run-id prefix: Cloud has no prefix lookup +AssertionError: expected [Function] to throw error matching /not full Cloud run ids: c649fe14/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/not full Cloud run ids: c649fe14/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[10/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses the whole batch when any id is invalid, rather than dropping it +AssertionError: expected [Function] to throw error matching /not full Cloud run ids: nope!/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/not full Cloud run ids: nope!/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[11/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses an empty batch +AssertionError: expected [Function] to throw error matching /needs runIds/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/needs runIds/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[12/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses a batch too large for the edge step lease +AssertionError: expected [Function] to throw error matching /exceeds the 8 that fit/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/exceeds the 8 that fit/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[13/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > accepts eight ids — the incident batch is inside the bound +AssertionError: promise rejected "TypeError: expected an @relayflows/surfac…" instead of resolving + ❯ tests/stuck-run-triage.test.ts:62:40 + 60| it('accepts eight ids — the incident batch is inside the bound', asy… + 61| const ids = Array.from({ length: 8 }, (_, i) => `${ID_A.slice(0, -… + 62| await expect(drive({ runIds: ids })).resolves.toBeDefined(); + | ^ + 63| }); + 64| }); + +Caused by: TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + ❯ tests/stuck-run-triage.test.ts:62:18 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[14/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > refuses to send the Cloud bearer token to an unapproved origin +AssertionError: expected [Function] to throw error matching /refusing to send the Cloud bearer to…/\ but got 'expected an @relayflows/surface flow …' + +- Expected: +/refusing to send the Cloud bearer token to https:\/\/evil\.example/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[15/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > refuses a non-URL apiUrl +AssertionError: expected [Function] to throw error matching /is not a URL/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/is not a URL/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[16/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > allows an approved origin and uses it in the curl +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:77:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[17/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > defaults to production Cloud +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:82:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[18/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > never publishes a run record the fetch did not produce +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:89:34 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[19/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > names the Worker on every wrangler invocation +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:98:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[20/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > accepts caller-supplied Workers and rejects option-shaped ones +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:107:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[21/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > falls back when GNU timeout is absent, as it is on macOS +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:115:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[22/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > runs the tails concurrently so wall time does not scale with the batch +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:123:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[23/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > records wrangler's own exit status rather than head's +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:129:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[24/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage shell text > parses under both sh and bash +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:137:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[25/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage shell text > collects tails with no GNU timeout on PATH, as on a stock macOS +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:157:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[26/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage agents > declares read-only permissions on every agent +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:176:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[27/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage agents > tells the forensics agents their evidence is untrusted +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:182:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[28/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > refuses a duplicate run id: two tails would share one evidence file +AssertionError: expected [Function] to throw error matching /duplicate runIds: c649fe14-0c2e-4e51-…/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/duplicate runIds: c649fe14-0c2e-4e51-9a6a-4f0d1b0f77aa/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[29/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > refuses a duplicate Worker name for the same reason +AssertionError: expected [Function] to throw error matching /duplicate workers: w-one/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/duplicate workers: w-one/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[30/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > bounds ids x workers, not just ids +AssertionError: expected [Function] to throw error matching /24 concurrent tails, over the 16/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/24 concurrent tails, over the 16/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[31/31]⎯ + + Test Files 3 failed (3) + Tests 30 failed | 23 passed | 14 skipped (67) + Start at 17:43:23 + Duration 54.32s (transform 839ms, setup 0ms, collect 2.22s, tests 53.21s, environment 0ms, prepare 120ms) + +EXIT=1 diff --git a/evidence/511-named-gate-diagnostics/head-package-test.txt b/evidence/511-named-gate-diagnostics/head-package-test.txt new file mode 100644 index 000000000..22ac5b54a --- /dev/null +++ b/evidence/511-named-gate-diagnostics/head-package-test.txt @@ -0,0 +1,935 @@ +# Required package command at HEAD (f308f95), run from packages/sdk. +# RELAYFLOWD_BIN is passed explicitly because test:prep exports it in a subshell. +$ git rev-parse HEAD +f308f95683c46eaddcf5c22eefd35aa04d2d1500 +$ git status --porcelain -- packages/sdk +$ RELAYFLOWD_BIN=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd npm test + +> @relayflows/sdk@2.0.22 test +> sh scripts/test.sh + + +> @relayflows/sdk@2.0.22 test:prep +> ( cd ../../kernel && sh ../ops/cargo.sh build ) && ( [ ! -d ../../testdata/preflight ] || find ../../testdata/preflight -name '*-cli' -type f -exec chmod +x {} + ) + + Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.07s + +> @relayflows/sdk@2.0.22 typecheck +> tsc --noEmit && tsc -p tsconfig.type-tests.json + + +> @relayflows/sdk@2.0.22 build +> tsc && node scripts/make-cli-executable.mjs + + +> @relayflows/sdk@2.0.22 typecheck:tests +> tsc -p tsconfig.tests.json + + + RUN v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk + +stdout | tests/live-kernel.test.ts +LIVE_KERNEL relayflowd=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd +LIVE_KERNEL flows=/home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/dist/cli.js + + ✓ tests/cloud-read.test.ts (39 tests) 73ms + ✓ tests/preflight.test.ts (57 tests) 92ms + ✓ tests/cli.test.ts (65 tests) 1415ms + ✓ flows check CLI > binds a checked relative wrapper to the flow directory for worker execution 375ms + ✓ tests/cloud-sync.test.ts (40 tests) 706ms + ✓ tests/observer-link.test.ts (39 tests) 146ms + ✓ tests/agent-transcript.test.ts (29 tests) 269ms + ✓ tests/cloud-run.test.ts (58 tests) 682ms +(node:84245) ExperimentalWarning: SQLite is an experimental feature and might change at any time +(Use `node --trace-warnings ...` to show where the warning was created) + ✓ tests/cli-status.test.ts (26 tests) 901ms + ✓ flows status > resolves the run with no arguments from inside a worker-spawned agent 711ms + ✓ tests/relay-cli-surface.test.ts (66 tests) 24ms + ✓ tests/authored-flow.test.ts (25 tests) 726ms + ✓ tests/daemon-lifecycle.test.ts (42 tests) 35ms + ✓ tests/run-state.test.ts (21 tests) 11ms + ✓ tests/cloud-deploy.test.ts (40 tests) 1072ms + ✓ tests/cloud-connect.test.ts (24 tests) 3290ms + ✓ hosted verbs connect before they submit > flows run --cloud submits once the prompt connected the integration 2152ms + ✓ tests/step-failure-diagnostic.test.ts (21 tests) 36ms + ✓ tests/worker-cli.test.ts (18 tests) 25255ms + ✓ registered CLI model defaults > passes the same priced Claude default to the real provider invocation 397ms + ✓ step discovery environment > names the run, step, attempt and an absolute data dir for a direct agent spawn 404ms + ✓ step discovery environment > exports none of the four without a data dir, even when the worker inherited them 334ms + ✓ wrapper discovery environment > exports none of the four to a wrapper without a data dir, even when the worker inherited them 365ms + ✓ custom wrapper execution identity > passes an explicit safe environment at identification and execution 346ms + ✓ custom wrapper execution identity > refuses a wrapper symlink retarget before delivering private values 408ms + ✓ custom wrapper execution identity > bounds wrapper execution after acknowledgement 426ms + ✓ custom wrapper execution identity > bounds captured wrapper output 342ms + ✓ custom wrapper execution identity > refuses a duplicate execute protocol frame 353ms + ✓ custom wrapper execution bounds are reader-owned > resolves when a conforming wrapper leaks a stdio pipe to a background helper 1938ms + ✓ custom wrapper execution bounds are reader-owned > resolves when the leaked helper inherits stderr only 1911ms + ✓ custom wrapper execution bounds are reader-owned > resolves when a wrapper leaks a stdio pipe and exits before identifying 3689ms + ✓ custom wrapper execution bounds are reader-owned > journals a completionReason at the default bound when a wrapper leaks a stdio pipe 11627ms + ✓ custom wrapper execution bounds are reader-owned > accepts an execute token and an over-8KiB payload flushed in one write 410ms + ✓ custom wrapper execution bounds are reader-owned > accepts the same over-8KiB payload whether or not it coalesces with the execute token 1185ms + ✓ custom wrapper execution bounds are reader-owned > still bounds an un-terminated handshake buffer and names the bound 377ms + ✓ delivers the journaled memory pack to the real wrapper and excludes its charge from completion usage 458ms + ❯ tests/authored-node-runtime.test.ts (14 tests | 14 skipped) 13ms + ✓ tests/close-pr-flow.test.ts (28 tests) 332ms + ✓ tests/journal-client.test.ts (15 tests) 77ms + ✓ tests/validate.test.ts (68 tests) 20ms + ✓ tests/verb-field-lint.test.ts (96 tests) 271ms + ✓ tests/authored-root.test.ts (12 tests) 175ms + ✓ tests/mcp.test.ts (30 tests) 20250ms + ✓ MCP preflight and transports > flows check refuses an undeclared server with exit 2 and no daemon 545ms + ✓ MCP preflight and transports > flows check reports a refusing server and leaves no PID 543ms + ✓ MCP preflight and transports > kills a SIGTERM-resistant silent child after a parent-owned handshake deadline 1319ms + ✓ MCP preflight and transports > reaps a SIGTERM-resistant descendant with inherit stdio before cleanup finishes 1121ms + ✓ MCP preflight and transports > reaps a SIGTERM-resistant descendant with ignore stdio before cleanup finishes 2065ms + ✓ MCP preflight and transports > reports malformed connection configuration as config_invalid 542ms + ✓ authored MCP effects against the real kernel > reports a dropped tool connection as a failed CLI run 13173ms + ✓ tests/tick-source.test.ts (33 tests) 23ms + ✓ tests/agent-relay-transport.test.ts (16 tests) 2220ms + ✓ Relay completion at the journal boundary > does not complete at readiness and journals exact output, receipt, and priced accounting 1005ms + ✓ Relay completion at the journal boundary > aborts polling on rejected renewal and never writes a stale completion 1003ms + ✓ tests/authored-flow-lifecycle-executor.test.ts (27 tests) 605ms + ✓ tests/pr-review-post.test.ts (21 tests) 2462ms + ✓ tests/flow-executor-chain.test.ts (14 tests) 10570ms + ✓ flow executor LLM and output-binding chain > runs f.llm -> f.agent -> f.run with schema-verified journal output and the exact allowed model 1018ms + ✓ flow executor LLM and output-binding chain > runs a dollar-budgeted authored Claude agent with the same default used by preflight 614ms + ✓ flow executor LLM and output-binding chain > preserves JSON values without promoting them to process wrappers: [1,2] 332ms + ✓ flow executor LLM and output-binding chain > preserves JSON values without promoting them to process wrappers: "hello" 365ms + ✓ flow executor LLM and output-binding chain > runs the exact authored flagship f.llm -> f.agent -> f.run path through the durable CLI root 1512ms + ✓ flow executor LLM and output-binding chain > resumes an interrupted durable authored root without replaying completed flagship effects 3531ms + ✓ flow executor LLM and output-binding chain > passes a declarative verified value through an agent into a deterministic artifact 886ms + ✓ flow executor LLM and output-binding chain > journals a missing optional field as a failure before the consuming command executes 378ms + ✓ flow executor LLM and output-binding chain > flows run consumes YAML bindings and resume reuses the original journal output 914ms + ✓ tests/authored-flow-slack.test.ts (7 tests) 1633ms + ✓ authored Slack helper effects > replays after SIGKILL before confirm with the same token and one successful completion 493ms + ✓ authored Slack helper effects > replays after SIGKILL before complete with the same token and one successful completion 474ms +(node:86295) ExperimentalWarning: SQLite is an experimental feature and might change at any time +(Use `node --trace-warnings ...` to show where the warning was created) + ✓ tests/cli-replay.test.ts (37 tests) 1011ms + ✓ flows replay > --json is byte-identical across two CLI invocations (diff) 685ms + ✓ tests/tick-runner.test.ts (22 tests) 2093ms + ✓ CLI argument parsing refuses coercion rather than accepting it > refuses --interval-ms fractional as an invocation error 344ms + ✓ CLI argument parsing refuses coercion rather than accepting it > refuses --interval-ms exponent notation as an invocation error 339ms + ✓ CLI argument parsing refuses coercion rather than accepting it > refuses --interval-ms hex as an invocation error 338ms + ✓ CLI argument parsing refuses coercion rather than accepting it > refuses --interval-ms trailing text as an invocation error 331ms + ✓ CLI argument parsing refuses coercion rather than accepting it > refuses --interval-ms empty as an invocation error 363ms + ✓ CLI argument parsing refuses coercion rather than accepting it > accepts an exact integer and proceeds past parsing 350ms +stdout | tests/live-kernel.test.ts > surface resume after a real daemon kill > resumes a three-step run with each successful completion exactly once +LIVE_KERNEL kill -9 pid=87028 run=01M2ZZ0GYBQ2EHWGJW8AXSB9E2 while step=two state=Running + + ✓ tests/stop-process-group.test.ts (6 tests) 6406ms + ✓ every stop reaches the process group, not just the direct child > exits the run after an execution-timeout stop 852ms + ✓ every stop reaches the process group, not just the direct child > exits the run after a protocol terminate stop 463ms + ✓ every stop reaches the process group, not just the direct child > kills a SIGTERM-deaf grandchild after a protocol terminate stop 1630ms + ✓ every stop reaches the process group, not just the direct child > kills a SIGTERM-deaf grandchild after an execution-timeout stop 2118ms + ✓ every stop reaches the process group, not just the direct child > holds the loop open long enough for the escalation to run 1082ms + ✓ tests/gate-contract.test.ts (20 tests) 118ms + ❯ tests/live-kernel.test.ts (31 tests | 8 failed) 53047ms + ✓ built flows CLI against live relayflowd > twenty-six-step reuses 25 durable completions after editing the failed final step 2008ms + ✓ built flows CLI against live relayflowd > runs rung (a), parks rung (b), and keeps JSON report-shaped 2400ms + ✓ built flows CLI against live relayflowd > allows a deterministic run to exceed the bounded request timeout 32417ms + ✓ built flows CLI against live relayflowd > follows a live worker dispatch through flows run 539ms + ✓ built flows CLI against live relayflowd > runs an agent CLI end to end through the SDK worker 563ms + ✓ built flows CLI against live relayflowd > f.agent lowers to a real agent step and dispatches through a live worker 584ms + ✓ built flows CLI against live relayflowd > can always get a parked run to a late-attaching worker 5578ms + ✓ built flows CLI against live relayflowd > reports a real manual-recovery NeedsHuman state as parked 521ms + × built flows CLI against live relayflowd > runs hn-monitor analyze-story end-to-end via a stub agent CLI (gate 2 clause 2 demo) 558ms + → expected { …(12) } to match object { output: { …(3) }, …(1) } +(22 matching properties omitted from actual) + × built flows CLI against live relayflowd > hn-monitor analyze-story FAILS verification when the CLI omits required schema fields 501ms + → expected { …(12) } to match object { …(3) } +(21 matching properties omitted from actual) + × built flows CLI against live relayflowd > agent step preserves the CliResult wrapper as output when the CLI emits non-JSON text 525ms + → expected null not to be null + × built flows CLI against live relayflowd > AgentWorker exposes wake_context to the CLI via RELAYFLOW_WAKE_CONTEXT env var (real analyzer prerequisite) 525ms + → Cannot read properties of null (reading 'story_title') + × built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_WAKE_CONTEXT UNSET when the run has no wake_context (undefined-vs-null pin) 501ms + → Cannot read properties of null (reading 'env_present') + ✓ built flows CLI against live relayflowd > AgentWorker passes a declared model to an identified wrapper as RELAYFLOW_MODEL 426ms + ✓ built flows CLI against live relayflowd > AgentWorker refuses a nonconforming journal-submitted wrapper before exposing RELAYFLOW_MODEL 512ms + ✓ built flows CLI against live relayflowd > AgentWorker executes the raw claude adapter with its real model flag 458ms + ✓ built flows CLI against live relayflowd > AgentWorker executes the raw codex adapter with its real model flag 487ms + × built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_MODEL UNSET when the step declares no model 529ms + → Cannot read properties of null (reading 'story_title') + × built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI 37ms + → LIVE_ANALYZER_UNAVAILABLE: "/home/daytona/.relayflow-v2-supervisor/durable/repository/testdata/preflight/analyze-story-claude-cli" does not identify as relayflows-agent-cli-v1 — failing because gate-2 acceptance requires the real analyzer to execute. Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is not gate evidence. + ✓ built flows CLI against live relayflowd > preflights before journaling and names an unreachable socket 746ms + ✓ built flows CLI against live relayflowd > starts exactly one daemon when two runs race for one empty data dir 481ms + ✓ surface resume after a real daemon kill > resumes a three-step run with each successful completion exactly once 805ms + × a relayflow can be scheduled: tick source against live relayflowd > a tick spawns a real run whose step reports the SCHEDULED instant 609ms + → expected null to deeply equal { schedule_id: 'heartbeat-1m', …(3) } + ✓ tests/authored-human.test.ts (13 tests) 90ms + ✓ tests/bundle.test.ts (23 tests) 8576ms + ✓ immutable bundles > returns exit 2 naming a byte-flipped payload and refuses to reuse corruption 352ms + ✓ immutable bundles > verifies with --verify in any position and answers --json with one object 697ms + ✓ immutable bundles > refuses --out with --verify rather than ignoring the destination 353ms + ✓ immutable bundles > builds and verifies the canonical YAML fixture through the compiled CLI 1037ms + ✓ immutable bundles > emits the ephemeral warning on CLI stderr and uses the default output directory 713ms + ✓ immutable bundles > refuses build-provable CLI resolution errors without environment probes 351ms + ✓ immutable bundles > builds a standalone TS fixture twice with identical executable hashes 2708ms + ✓ immutable bundles > refuses to label installed dependency drift with lockfile pins 346ms + ✓ immutable bundles > refuses invalid CLI arguments %j 347ms + ✓ immutable bundles > refuses invalid CLI arguments "--out" 370ms + ✓ immutable bundles > refuses invalid CLI arguments "--verify" 359ms + ✓ immutable bundles > refuses invalid CLI arguments "--verify" 373ms + ✓ immutable bundles > refuses invalid CLI arguments "--out" 459ms + ✓ tests/cli-hn-monitor.test.ts (16 tests) 93ms + ✓ tests/named-gate-diagnostics.test.ts (17 tests) 532ms + ✓ tests/cloud-schedule.test.ts (17 tests) 4003ms + ✓ schedule lowering > marks a non-grid cron as Cloud-only rather than approximating it, with a silence budget from its own cadence 1705ms + ✓ flows check prints declared schedules > shows the lowering for a fixed interval and the Cloud-only note for a real cron 1616ms + ✓ tests/authored-node-result.test.ts (38 tests) 39ms + ✓ tests/direct-input.test.ts (6 tests) 5342ms + ✓ direct .flow.ts input through the built CLI and live runtime > returns exit 3 for an authored human handoff and persists its outcome 634ms + ✓ direct .flow.ts input through the built CLI and live runtime > returns exit 1 for an authored step_failed verdict and persists its outcome 583ms + ✓ direct .flow.ts input through the built CLI and live runtime > executes inline and file JSON input through relayflowd 1770ms + ✓ direct .flow.ts input through the built CLI and live runtime > refuses missing and malformed input before contacting relayflowd 1430ms + ✓ direct .flow.ts input through the built CLI and live runtime > does not run the authored body before daemon availability 509ms + ✓ direct .flow.ts input through the built CLI and live runtime > refuses oversized file input before contacting relayflowd 415ms + ✓ tests/daemon-lifecycle-live.test.ts (9 tests) 6385ms + ✓ flows run against a data dir with no daemon (§6 test 7) > cold start spawns exactly one daemon, the run succeeds, and the daemon outlives the CLI 409ms + ✓ flows run against a data dir with no daemon (§6 test 7) > polls, bounded, for a daemon that holds the lock before it binds 1334ms + ✓ flows run against a data dir with no daemon (§6 test 7) > attaches to a serving daemon that has not published a connection file 524ms + ✓ flows run against a data dir with no daemon (§6 test 7) > a second run attaches to the daemon the first one started, spawning nothing 974ms + ✓ flows run against a data dir with no daemon (§6 test 7) > detects a stale connection file left by a hard kill and starts a fresh daemon 873ms + ✓ concurrent invocations against one empty data dir (§6 test 15) > ends with exactly one daemon owning the socket, and both runs succeed 1101ms + ✓ refusals from a spawn that cannot produce a daemon > names relayflowd_not_found rather than falling through to PATH 359ms + ✓ refusals from a spawn that cannot produce a daemon > names daemon_start_failed and quotes the daemon log when startup dies 413ms + ✓ refusals from a spawn that cannot produce a daemon > refuses a daemon speaking another protocol version instead of binding over it 397ms +(node:88133) Warning: Transcript tail for run-9/analyze attempt 1 (stdout) could not be written; the step continues without it: EACCES: permission denied, mkdir '/tmp/transcript-tail-c9FNaN/runs/run-9/steps' +(Use `node --trace-warnings ...` to show where the warning was created) + ✓ tests/transcript-tail.test.ts (11 tests) 831ms + ✓ direct agent spawn > completes the step when the tail directory cannot be created 442ms + ✓ tests/authored-agent-artifacts.test.ts (4 tests) 469ms + ✓ tests/authored-helpers.test.ts (6 tests) 3034ms + ✓ runs every available provider through the real kernel and resumes completed effects without a second write 1653ms + ✓ replays after SIGKILL before confirm with the same token and one successful completion 493ms + ✓ replays after SIGKILL before complete with the same token and one successful completion 471ms + ✓ tests/backlog-picker.test.ts (14 tests) 40ms + ✓ tests/backlog-picker-flow.test.ts (6 tests) 316ms + ✓ tests/preflight-permissions-unenforced.test.ts (17 tests) 261ms + ❯ tests/stuck-run-triage.test.ts (22 tests | 22 failed) 63ms + × stuck-run-triage input validation > refuses an 8-character run-id prefix: Cloud has no prefix lookup 4ms + → expected [Function] to throw error matching /not full Cloud run ids: c649fe14/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses the whole batch when any id is invalid, rather than dropping it 0ms + → expected [Function] to throw error matching /not full Cloud run ids: nope!/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses an empty batch 0ms + → expected [Function] to throw error matching /needs runIds/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > refuses a batch too large for the edge step lease 0ms + → expected [Function] to throw error matching /exceeds the 8 that fit/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage input validation > accepts eight ids — the incident batch is inside the bound 3ms + → promise rejected "TypeError: expected an @relayflows/surfac…" instead of resolving + × stuck-run-triage apiUrl > refuses to send the Cloud bearer token to an unapproved origin 0ms + → expected [Function] to throw error matching /refusing to send the Cloud bearer to…/\ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage apiUrl > refuses a non-URL apiUrl 0ms + → expected [Function] to throw error matching /is not a URL/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage apiUrl > allows an approved origin and uses it in the curl 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage apiUrl > defaults to production Cloud 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage apiUrl > never publishes a run record the fetch did not produce 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > names the Worker on every wrangler invocation 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > accepts caller-supplied Workers and rejects option-shaped ones 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > falls back when GNU timeout is absent, as it is on macOS 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > runs the tails concurrently so wall time does not scale with the batch 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage edge collection > records wrangler's own exit status rather than head's 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage shell text > parses under both sh and bash 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage shell text > collects tails with no GNU timeout on PATH, as on a stock macOS 48ms + → expected an @relayflows/surface flow handle + × stuck-run-triage agents > declares read-only permissions on every agent 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage agents > tells the forensics agents their evidence is untrusted 0ms + → expected an @relayflows/surface flow handle + × stuck-run-triage fan-out > refuses a duplicate run id: two tails would share one evidence file 0ms + → expected [Function] to throw error matching /duplicate runIds: c649fe14-0c2e-4e51-…/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage fan-out > refuses a duplicate Worker name for the same reason 1ms + → expected [Function] to throw error matching /duplicate workers: w-one/ but got 'expected an @relayflows/surface flow …' + × stuck-run-triage fan-out > bounds ids x workers, not just ids 0ms + → expected [Function] to throw error matching /24 concurrent tails, over the 16/ but got 'expected an @relayflows/surface flow …' + ✓ tests/worker-transcript.test.ts (5 tests) 187ms + ✓ tests/flow-requirements.test.ts (13 tests) 691ms + ✓ flows check prints REQUIRES > names the helper, the harness and the mcp server of an authored flow 446ms + ✓ tests/webhook.test.ts (9 tests) 500ms + ✓ webhook ingress > checks TS declarations against flows.json without invoking handlers 431ms + ✓ tests/authored-run-failure-evidence.test.ts (8 tests) 747ms + ✓ the child index after the process that wrote it is gone > still names every child, with its own run id, after a daemon restart 441ms + ✓ tests/webhook-live.test.ts (6 tests) 9547ms + ✓ executes and deduplicates 'app_mention' only for its provider and matching payload 1453ms + ✓ executes and deduplicates 'reaction_added' only for its provider and matching payload 1398ms + ✓ executes and deduplicates 'pull_request' only for its provider and matching payload 1415ms + ✓ flows serve-webhook writes JSON before the daemon starts, then journals and archives exactly once 1407ms + ✓ replays a dropped file after SIGKILL before spawn 435ms + ✓ resumes the same journal after SIGKILL after spawn and before acknowledgement 3438ms + ✓ tests/cli-watch.test.ts (10 tests) 14714ms + ✓ flows check --watch > rechecks syntax errors, clears once, and returns the last refusal on Ctrl-C 1209ms + ✓ flows check --watch > streams JSON lines without ANSI, recovers after atomic saves, and exits zero after repair 1712ms + ✓ flows check --watch > coalesces 20 concurrent saves into at most two rechecks 1721ms + ✓ flows check --watch > watches transitive relative use imports, cycles, and nearest config changes 2213ms + ✓ flows check --watch > refreshes the import graph and notices missing imports being created 2215ms + ✓ flows check --watch > reloads authored TypeScript instead of reusing the first imported definition 1454ms + ✓ flows check --watch > detects a nearer config appearing and falls back after it is deleted 1700ms + ✓ flows check --watch > keeps watching after the target is deleted and recreated 1704ms + ✓ flows check --watch > queues changes during a slow check without overlapping checks 781ms + ✓ tests/agent-transcript-live.test.ts (4 tests) 41163ms + ✓ the transcript digest through the built CLI, a real daemon and the local agent > preserves structured agent failure details and its completed root index 12627ms + ✓ the transcript digest through the built CLI, a real daemon and the local agent > preserves structured llm failure details and its completed root index 14532ms + ✓ the transcript digest through the built CLI, a real daemon and the local agent > journals the digest in trajectory_tail on a successful agent step and writes the file it points at 805ms + ✓ the transcript digest through the built CLI, a real daemon and the local agent > on a failed agent step, names the failure and the transcript in the terminal diagnostic, redacted 13198ms + ✓ tests/authored-step-failed.test.ts (10 tests) 37ms + ✓ tests/authored-flow-operation.test.ts (23 tests) 382ms + ✓ tests/agent-artifacts-live.test.ts (5 tests) 44324ms + ✓ agent artifacts and gates through the built CLI, a real daemon and the local agent > journals the files the agent wrote, and both artifact gates pass on that journal 1031ms + ✓ agent artifacts and gates through the built CLI, a real daemon and the local agent > fails the run when the artifact_exists gate names a file the agent did not write 14623ms + ✓ agent artifacts and gates through the built CLI, a real daemon and the local agent > fails the run with the author reason when a predicate gate returns false, journaling the verdict 13955ms + ✓ review follow-ups > applies a predicate gate on a helper step too, and journals its verdict 14047ms + ✓ review follow-ups > records predicate verdicts on the root run so a resume reuses them instead of re-running the closure 667ms + ✓ tests/budget-preflight.test.ts (25 tests) 13ms + ✓ tests/authored-step-index.test.ts (12 tests) 11ms + ✓ tests/artifact-gates.test.ts (6 tests) 119ms + ✓ tests/human-live.test.ts (3 tests) 5799ms + ✓ f.human against a real daemon > parks with the question, refuses wrong answers, records one, and resumes to success 3362ms + ✓ f.human against a real daemon > a "no" is a value the body branches on: declined, exit 0, no effect 1547ms + ✓ f.human against a real daemon > refuses to answer a run the daemon does not know 889ms + ✓ tests/provider-trigger-contract.test.ts (7 tests) 586ms + ✓ provider trigger contract > fails `flows check` before deployment and passes once the event is real 378ms + ✓ tests/work-package-consumer.test.ts (13 tests) 132ms + ✓ tests/budget-unmetered-live.test.ts (3 tests) 1273ms + ✓ unmetered budget spend through the live kernel > runs an unpriced step under a dollar budget without tripping it, journaling unknown dollars 506ms + ✓ unmetered budget spend through the live kernel > still counts an unpriced step toward a token budget 403ms + ✓ unmetered budget spend through the live kernel > accrues a priced step and stops the run when it crosses the dollar budget 363ms + ✓ tests/helpers-fanout.test.ts (96 tests) 145ms + ✓ tests/spec-parity.test.ts (31 tests) 307ms + ✓ tests/generate-triggers.test.ts (7 tests) 1030ms + ✓ discovers new adapters, preserves exact event names, and prefers adapter-local mappings 327ms + ✓ tests/named-gate-journal.test.ts (5 tests) 1959ms + ✓ a lowered subprocess_gate journals the gate command's streams > persists what the gate printed before a timeout killed it 822ms + ✓ a gate on an agent step > journals the command's streams and surfaces them in the authored failure 834ms + ✓ tests/webhook-hardening.test.ts (11 tests) 54ms + ✓ tests/human-to.test.ts (8 tests) 9ms + ✓ tests/plugin-loader.test.ts (9 tests) 165ms + ✓ tests/pty-sidechannel.test.ts (11 tests) 6628ms + ✓ view attach preserves worker completion and marks only drive 906ms + ✓ drive attach preserves worker completion and marks only drive 451ms + ✓ passthrough attach preserves worker completion and marks only drive 899ms + ✓ none attach preserves worker completion and marks only drive 885ms + ✓ none subscriber lets an unattended CLI read EOF 531ms + ✓ view subscriber lets an unattended CLI read EOF 473ms + ✓ passthrough subscriber lets an unattended CLI read EOF 464ms + ✓ incomplete subscriber lets an unattended CLI read EOF 515ms + ✓ rejects drive after EOF without marking human intervention 819ms + ✓ delivers all drive bytes in order across child stdin backpressure 679ms + ✓ tests/worker-lease.test.ts (7 tests) 10ms + ✓ tests/yaml-helpers.test.ts (33 tests) 63ms + ✓ tests/authored-agent-permissions.test.ts (26 tests) 782ms + ✓ tests/deploy.test.ts (11 tests) 4627ms + ✓ flows deploy file buckets > publishes the full signed layout byte-for-byte and redeploys as a noop 735ms + ✓ flows deploy file buckets > answers --json with one object per outcome 716ms + ✓ flows deploy file buckets > reports a refusal as JSON under --json 343ms + ✓ flows deploy file buckets > refuses a missing local bundle before creating the bucket 349ms + ✓ flows deploy file buckets > refuses an unreachable bucket before copying 352ms + ✓ flows deploy file buckets > refuses an unwritable bucket 342ms + ✓ flows deploy file buckets > refuses local tampering of spec.canonical.json 336ms + ✓ flows deploy file buckets > refuses local tampering of identity.json 342ms + ✓ flows deploy file buckets > refuses asset bundles instead of using daemon-relative files 371ms + ✓ flows deploy file buckets > never labels a corrupt existing deployment as a noop 720ms + ✓ tests/redact.test.ts (35 tests) 8ms + ✓ tests/communication.test.ts (10 tests) 13ms + ✓ tests/typed-output.test.ts (14 tests) 180ms + ✓ tests/budget-attribution.test.ts (5 tests) 6ms + ✓ tests/json-schema-bound.test.ts (71 tests) 2113ms + ✓ JSON Schema termination bound > walks a deep schema with an explicit stack rather than recursion 1724ms + ✓ tests/effect-channel.test.ts (5 tests) 378ms + ✓ tests/mcp-lifecycle.test.ts (4 tests) 13ms + ✓ tests/model-selection.test.ts (10 tests) 15ms + ✓ tests/f-memory.test.ts (7 tests) 955ms + ✓ tests/relayflowd-path.test.ts (10 tests) 6ms + ✓ tests/worker-cli-result-exit.test.ts (5 tests) 32907ms + ✓ a Claude agent step completes on its result, not only on process exit > settles a hung, successful run within the grace and stops its whole tree 31611ms + ✓ a Claude agent step completes on its result, not only on process exit > maps an error result on a hung run to a failed exit 31612ms + ✓ a Claude agent step completes on its result, not only on process exit > leaves a hang before any result to the existing stops 32013ms + ✓ an agent tree does not outlive the process that spawned it > kills the agent group when the run process is terminated by SIGTERM 800ms + ✓ tests/authored-plugin-effect.test.ts (6 tests) 49ms + ✓ tests/yaml-local-agent-live.test.ts (7 tests) 3806ms + ✓ YAML --local-agent through the built CLI and real daemon > runs with the checked step CLI and model and journals done 541ms + ✓ YAML --local-agent through the built CLI and real daemon > runs with the checked named CLI and model and journals done 564ms + ✓ YAML --local-agent through the built CLI and real daemon > runs with the checked flow CLI and model and journals done 557ms + ✓ YAML --local-agent through the built CLI and real daemon > runs with the checked project CLI and model and journals done 576ms + ✓ YAML --local-agent through the built CLI and real daemon > still parks without --local-agent 494ms + ✓ YAML --local-agent through the built CLI and real daemon > reports the agent process failure 575ms + ✓ YAML --local-agent through the built CLI and real daemon > preserves declared workspace surfaces that the local worker cannot pin 498ms + ✓ tests/local-dev-ux.test.ts (8 tests) 18ms + ✓ tests/relay-cli-surface-live.test.ts (3 tests) 467ms + ✓ tests/authored-declined.test.ts (13 tests) 56ms + ✓ tests/resume-failure.test.ts (2 tests) 5ms + ✓ tests/dependency-validation.test.ts (6 tests) 577ms + ✓ dependency validation > accepts a valid 10,000-step reverse chain through every direct public boundary 322ms + ✓ tests/input-binding.test.ts (12 tests) 171ms + ✓ tests/communication-review.test.ts (5 tests) 332ms + ✓ tests/yaml-helper-effect.test.ts (4 tests) 75ms + ✓ tests/deterministic-llm.test.ts (5 tests) 47ms + ✓ tests/scope-preflight.test.ts (6 tests) 7ms + ✓ tests/bin.test.ts (7 tests) 2178ms + ✓ built flows binary > refuses through a symlink to the built artifact 344ms + ✓ built flows binary > refuses through a symlinked directory component 333ms + ✓ built flows binary > classifies a signal-terminated auth probe as probe_failed 449ms + ✓ built flows binary > classifies an unavailable PATH resolver as probe_failed 358ms + ✓ built flows binary > does not describe a present non-executable CLI as missing 344ms + ✓ built flows binary > runs one auth probe for three steps sharing a flow CLI 349ms + ✓ tests/build-gate.test.ts (3 tests) 1046ms + ✓ flows build gates on flows check green (#318) > refuses a flow with an unresolvable named-agent CLI and leaves no artifacts 342ms + ✓ flows build gates on flows check green (#318) > --json emits one CheckReport object on stdout on refusal, exits 2, no artifacts 347ms + ✓ flows build gates on flows check green (#318) > builds the bundle on success (regression: gate must not block valid flows) 356ms + ✓ tests/scope-compiler.test.ts (25 tests) 11ms + ✓ tests/run-from-digest.test.ts (6 tests) 3922ms + ✓ flows run digest input > submits the sealed canonical spec through the normal journal path without checkout 398ms + ✓ flows run digest input > uses a verified cache hit even after the bucket is removed 369ms + ✓ flows run digest input > resolves deploy.bucket from flows.json and honors explicit override 1069ms + ✓ flows run digest input > refuses an unconfigured bucket 705ms + ✓ flows run digest input > refuses tampered spec.canonical.json before creating run data 704ms + ✓ flows run digest input > refuses tampered identity.json before creating run data 676ms + ✓ tests/communication-worker.test.ts (15 tests) 1497ms + ✓ tests/hn-poller.test.ts (6 tests) 7ms + ✓ tests/plugin-add.test.ts (7 tests) 1113ms + ✓ typechecks the augmented verb and rejects unknown namespaces 824ms + ✓ tests/authored-step-failed-exit.test.ts (3 tests) 7ms + ✓ tests/direct-run-failure.test.ts (8 tests) 11ms + ✓ tests/dir-watcher-poller.test.ts (6 tests) 5ms + ✓ tests/model-pricing.test.ts (10 tests) 5ms + ✓ tests/yaml-helper-live.test.ts (1 test) 893ms + ✓ runs compiled YAML helpers through the built CLI and kernel effect journal 892ms + ✓ tests/provider-trigger-executor.test.ts (4 tests) 287ms + ✓ tests/transcript-tail-close.test.ts (2 tests) 858ms + ✓ a stalled transcript-tail close > does not hold the spawn open past its bounded window 498ms + ✓ a stalled tail close beside a transcript that finished > still journals the transcript pointer 360ms + ✓ tests/wrapper-artifacts-cwd.test.ts (2 tests) 65ms + ✓ tests/hello-deterministic.test.ts (5 tests) 15ms + ✓ tests/transcript-exclusion-timeout.test.ts (1 test) 183ms + ✓ tests/cli-adapter.test.ts (4 tests) 5ms + ✓ tests/communication-mixed-resume.test.ts (1 test) 167ms + ✓ tests/work-package-validator.test.ts (7 tests) 5ms + ✓ tests/authored-use-loader.test.ts (5 tests) 819ms + ✓ tests/authored-declined-live.test.ts (1 test) 1508ms + ✓ runs an input guard and resumes its completed declined root without repeated effects 1507ms + ✓ tests/cli-answer.test.ts (15 tests) 8ms + ✓ tests/bundle-preflight.test.ts (4 tests) 791ms + ✓ bundle execution preflight > ignores surrounding cache configuration on a verified cache hit 370ms + ✓ bundle execution preflight > uses the built alias for a nameless flow even in a digest-only cache directory 391ms + ✓ tests/agent-relay-hardening.test.ts (12 tests) 12ms + ✓ tests/classify-outcome.test.ts (2 tests) 2160ms + ✓ classifyOutcome > gives up and reports when a running run never becomes classifiable 2007ms + ✓ tests/communication-preflight.test.ts (13 tests) 33ms + ✓ tests/agent-artifacts.test.ts (6 tests) 13ms + ↓ tests/real-cli-adapters.test.ts (3 tests | 3 skipped) + ✓ tests/memoization.test.ts (57 tests) 51ms + ✓ tests/parse-json-output.test.ts (7 tests) 3ms + ✓ tests/journal-client-completion.test.ts (4 tests) 99ms + ✓ tests/worker-cli-abort.test.ts (2 tests) 2542ms + ✓ stops claude and its process group when lease ownership is lost 1252ms + ✓ stops wrapper.mjs and its process group when lease ownership is lost 1289ms + ✓ tests/communication-environment-preflight.test.ts (6 tests) 3ms + ✓ tests/budget-authored-live.test.ts (2 tests) 158ms + ✓ tests/slack-writeback.test.ts (1 test) 257ms + ✓ tests/authored-surface-authority.test.ts (2 tests) 15ms + ✓ tests/adapters/claude.test.ts (7 tests) 4ms + ✓ tests/worker-cli-cwd.test.ts (2 tests) 351ms + ✓ runAgentCli — cwd propagation (flows#357) > omits cwd when not provided (inherits parent cwd) 346ms + ✓ tests/adapters/codex.test.ts (7 tests) 4ms + ✓ tests/slack-block-kit.test.ts (5 tests) 14ms + ✓ tests/communication-history.test.ts (1 test) 3ms + ✓ tests/adapters/registry.test.ts (4 tests) 4ms + ✓ tests/step-lease.test.ts (36 tests) 66424ms + ✓ f.run leases against the live kernel > enforces 10000 ms for 'sleep 5; printf ok' 5071ms + ✓ f.run leases against the live kernel > enforces 40000 ms for 'sleep 31; printf ok' 31064ms + ✓ f.run leases against the live kernel > enforces 30000 ms for 'sleep 31; printf ok' 30090ms + ✓ tests/authored-declined-report.test.ts (6 tests) 7ms + ✓ tests/communication-refusal.test.ts (1 test) 12ms + ✓ tests/check-command-cwd.test.ts (1 test) 12ms + ✓ tests/communication-lazy.test.ts (1 test) 3ms + ✓ tests/cli-progress-wait.test.ts (2 tests) 3ms + ✓ tests/bundle-transport.test.ts (20 tests) 2338ms + ✓ digest references > accepts and deploys the build output for hello 372ms + ✓ digest references > accepts and deploys the build output for Hello 360ms + ✓ digest references > accepts and deploys the build output for hello.world 378ms + ✓ digest references > accepts and deploys the build output for hello_world 434ms + ✓ digest references > accepts and deploys the build output for 123 431ms + ✓ digest references > accepts and deploys the build output for A_b.c-1 358ms + ✓ tests/run-digest-live.test.ts (1 test) 858ms + ✓ executes a deployed digest on the real kernel after deleting the authoring tree 857ms + ✓ tests/placement.test.ts (54 tests) 16ms + ✓ tests/communication-tools.test.ts (1 test) 80ms + ✓ tests/authored-admission.test.ts (2 tests) 2ms + ✓ tests/memory.test.ts (18 tests) 8ms + ✓ tests/worker-platform.test.ts (1 test) 3ms + ✓ tests/run-digest.test.ts (4 tests) 1391ms + ✓ digest run configuration refusals > reports config_invalid before fetching or starting a run for {invalid json 354ms + ✓ digest run configuration refusals > reports config_invalid before fetching or starting a run for {"deploy":{}} 330ms + ✓ digest run configuration refusals > reports config_invalid before fetching or starting a run for {"deploy":{"bucket":123}} 354ms + ✓ digest run configuration refusals > reports config_invalid before fetching or starting a run for {"deploy":{"bucket":""}} 353ms + ✓ tests/local-agent-live.test.ts (5 tests) 64632ms + ✓ built CLI local agent against a real daemon > dispatches through the wrapper and keeps --json stdout report-shaped 705ms + ✓ built CLI local agent against a real daemon > runs beyond the initial 30-second lease without a second invocation 35717ms + ✓ built CLI local agent against a real daemon > renders actual agent completion in text output 764ms + ✓ built CLI local agent against a real daemon > returns a failed run when the agent process fails 13068ms + ✓ built CLI local agent against a real daemon > refuses a workspace it cannot pin before invoking the agent 14377ms + +⎯⎯⎯⎯⎯⎯ Failed Suites 1 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/authored-node-runtime.test.ts [ tests/authored-node-runtime.test.ts ] +AssertionError: expected '1.3.6' to be '1.4.0' // Object.is equality + +Expected: "1.4.0" +Received: "1.3.6" + + ❯ tests/authored-node-runtime.test.ts:18:77 + 16| + 17| beforeAll(() => { + 18| expect(spawnSync(bun, ['--version'], { encoding: 'utf8' }).stdout.tr… + | ^ + 19| expect(existsSync(daemon), 'build the current kernel or set RELAYFLO… + 20| stage = mkdtempSync(join(tmpdir(), 'authored-standalone-build-')); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/31]⎯ + +⎯⎯⎯⎯⎯⎯ Failed Tests 30 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > runs hn-monitor analyze-story end-to-end via a stub agent CLI (gate 2 clause 2 demo) +AssertionError: expected { …(12) } to match object { output: { …(3) }, …(1) } +(22 matching properties omitted from actual) + +- Expected ++ Received + + Object { +- "output": Object { +- "reasoning": "stub agent runtime — deterministic output for gate-2 clause-2 demo", +- "relevance_score": 5, +- "story_title": "stub", +- }, ++ "output": null, + "verification": Object { +- "gate": "json_schema", +- "verdict": "pass", ++ "gate": "execution", ++ "verdict": "fail", + }, + } + + ❯ tests/live-kernel.test.ts:657:36 + 655| && (entry as { step_id?: string }).step_id === 'analyze-story', + 656| ) as { payload: { output: unknown; verification: unknown } } | und… + 657| expect(stepCompleted?.payload).toMatchObject({ + | ^ + 658| output: { + 659| story_title: 'stub', + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > hn-monitor analyze-story FAILS verification when the CLI omits required schema fields +AssertionError: expected { …(12) } to match object { …(3) } +(21 matching properties omitted from actual) + +- Expected ++ Received + + Object { +- "completionReason": "retries_exhausted", ++ "completionReason": "worker_error", + "output": null, + "verification": Object { +- "gate": "json_schema", ++ "gate": "execution", + "verdict": "fail", + }, + } + + ❯ tests/live-kernel.test.ts:752:36 + 750| // its verification record names the json_schema rejection. The re… + 751| // parsed value is nulled before the completion is persisted. + 752| expect(stepCompleted?.payload).toMatchObject({ + | ^ + 753| completionReason: 'retries_exhausted', + 754| output: null, + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[3/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > agent step preserves the CliResult wrapper as output when the CLI emits non-JSON text +AssertionError: expected null not to be null + ❯ tests/live-kernel.test.ts:823:24 + 821| // here (parseJsonOutput returned null on non-JSON stdout) and + 822| // these assertions would all fail. + 823| expect(output).not.toBeNull(); + | ^ + 824| expect(output.exit_code).toBe(0); + 825| expect(output.stdout_tail).toContain('looked at the story'); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[4/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker exposes wake_context to the CLI via RELAYFLOW_WAKE_CONTEXT env var (real analyzer prerequisite) +TypeError: Cannot read properties of null (reading 'story_title') + ❯ tests/live-kernel.test.ts:891:42 + 889| ) as { payload: { output: { story_title: string; reasoning: string… + 890| expect(stepCompleted).toBeDefined(); + 891| expect(stepCompleted!.payload.output.story_title).toBe(`echoed:${s… + | ^ + 892| expect(stepCompleted!.payload.output.reasoning).toContain(String(s… + 893| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[5/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_WAKE_CONTEXT UNSET when the run has no wake_context (undefined-vs-null pin) +TypeError: Cannot read properties of null (reading 'env_present') + ❯ tests/live-kernel.test.ts:958:38 + 956| ) as { payload: { output: { env_present: boolean } } } | undefined; + 957| expect(completed).toBeDefined(); + 958| expect(completed!.payload.output.env_present).toBe(false); + | ^ + 959| + 960| delete process.env.RELAYFLOW_WAKE_CONTEXT; + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[6/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > AgentWorker leaves RELAYFLOW_MODEL UNSET when the step declares no model +TypeError: Cannot read properties of null (reading 'story_title') + ❯ tests/live-kernel.test.ts:1194:38 + 1192| expect(completed).toBeDefined(); + 1193| // UNSET, not EMPTY and not the leaked parent value. + 1194| expect(completed!.payload.output.story_title).toBe('model:UNSET'); + | ^ + 1195| + 1196| delete process.env.RELAYFLOW_MODEL; + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[7/31]⎯ + + FAIL tests/live-kernel.test.ts > built flows CLI against live relayflowd > hn-monitor analyze-story reaches done through the real Claude analyzer CLI +Error: LIVE_ANALYZER_UNAVAILABLE: "/home/daytona/.relayflow-v2-supervisor/durable/repository/testdata/preflight/analyze-story-claude-cli" does not identify as relayflows-agent-cli-v1 — failing because gate-2 acceptance requires the real analyzer to execute. Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is not gate evidence. + ❯ tests/live-kernel.test.ts:1223:15 + 1221| const notice = `LIVE_ANALYZER_UNAVAILABLE: ${readiness.detail}`; + 1222| if (process.env['RELAYFLOWS_ALLOW_ANALYZER_SKIP'] !== '1') { + 1223| throw new Error( + | ^ + 1224| `${notice} — failing because gate-2 acceptance requires the … + 1225| + 'Set RELAYFLOWS_ALLOW_ANALYZER_SKIP=1 only if this run is … + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[8/31]⎯ + + FAIL tests/live-kernel.test.ts > a relayflow can be scheduled: tick source against live relayflowd > a tick spawns a real run whose step reports the SCHEDULED instant +AssertionError: expected null to deeply equal { schedule_id: 'heartbeat-1m', …(3) } + +- Expected: +Object { + "lag_ms": 43000, + "schedule_id": "heartbeat-1m", + "scheduled_for_ms": 1764000000000, + "slot": 29400000, +} + ++ Received: +null + + ❯ tests/live-kernel.test.ts:1665:39 + 1663| // The bound: the run reports the grid instant and its own lag, so… + 1664| // backfilled run can tell it is running for a slot from the past. + 1665| expect(completed!.payload.output).toEqual({ + | ^ + 1666| schedule_id: 'heartbeat-1m', + 1667| slot: 29_400_000, + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[9/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses an 8-character run-id prefix: Cloud has no prefix lookup +AssertionError: expected [Function] to throw error matching /not full Cloud run ids: c649fe14/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/not full Cloud run ids: c649fe14/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[10/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses the whole batch when any id is invalid, rather than dropping it +AssertionError: expected [Function] to throw error matching /not full Cloud run ids: nope!/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/not full Cloud run ids: nope!/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[11/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses an empty batch +AssertionError: expected [Function] to throw error matching /needs runIds/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/needs runIds/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[12/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > refuses a batch too large for the edge step lease +AssertionError: expected [Function] to throw error matching /exceeds the 8 that fit/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/exceeds the 8 that fit/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[13/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage input validation > accepts eight ids — the incident batch is inside the bound +AssertionError: promise rejected "TypeError: expected an @relayflows/surfac…" instead of resolving + ❯ tests/stuck-run-triage.test.ts:62:40 + 60| it('accepts eight ids — the incident batch is inside the bound', asy… + 61| const ids = Array.from({ length: 8 }, (_, i) => `${ID_A.slice(0, -… + 62| await expect(drive({ runIds: ids })).resolves.toBeDefined(); + | ^ + 63| }); + 64| }); + +Caused by: TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + ❯ tests/stuck-run-triage.test.ts:62:18 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[14/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > refuses to send the Cloud bearer token to an unapproved origin +AssertionError: expected [Function] to throw error matching /refusing to send the Cloud bearer to…/\ but got 'expected an @relayflows/surface flow …' + +- Expected: +/refusing to send the Cloud bearer token to https:\/\/evil\.example/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[15/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > refuses a non-URL apiUrl +AssertionError: expected [Function] to throw error matching /is not a URL/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/is not a URL/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[16/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > allows an approved origin and uses it in the curl +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:77:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[17/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > defaults to production Cloud +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:82:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[18/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage apiUrl > never publishes a run record the fetch did not produce +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:89:34 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[19/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > names the Worker on every wrangler invocation +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:98:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[20/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > accepts caller-supplied Workers and rejects option-shaped ones +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:107:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[21/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > falls back when GNU timeout is absent, as it is on macOS +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:115:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[22/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > runs the tails concurrently so wall time does not scale with the batch +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:123:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[23/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage edge collection > records wrangler's own exit status rather than head's +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:129:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[24/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage shell text > parses under both sh and bash +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:137:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[25/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage shell text > collects tails with no GNU timeout on PATH, as on a stock macOS +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:157:33 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[26/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage agents > declares read-only permissions on every agent +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:176:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[27/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage agents > tells the forensics agents their evidence is untrusted +TypeError: expected an @relayflows/surface flow handle + ❯ Module.getFlowDefinition node_modules/@relayflows/surface/src/flow.ts:143:11 + ❯ drive tests/stuck-run-triage.test.ts:34:9 + 32| done: () => {}, + 33| }; + 34| await getFlowDefinition(triage).body(f as never… + | ^ + 35| return rec; + 36| } + ❯ tests/stuck-run-triage.test.ts:182:23 + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[28/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > refuses a duplicate run id: two tails would share one evidence file +AssertionError: expected [Function] to throw error matching /duplicate runIds: c649fe14-0c2e-4e51-…/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/duplicate runIds: c649fe14-0c2e-4e51-9a6a-4f0d1b0f77aa/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[29/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > refuses a duplicate Worker name for the same reason +AssertionError: expected [Function] to throw error matching /duplicate workers: w-one/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/duplicate workers: w-one/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[30/31]⎯ + + FAIL tests/stuck-run-triage.test.ts > stuck-run-triage fan-out > bounds ids x workers, not just ids +AssertionError: expected [Function] to throw error matching /24 concurrent tails, over the 16/ but got 'expected an @relayflows/surface flow …' + +- Expected: +/24 concurrent tails, over the 16/ + ++ Received: +"expected an @relayflows/surface flow handle" + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[31/31]⎯ + + Test Files 3 failed | 155 passed | 1 skipped (159) + Tests 30 failed | 2393 passed | 17 skipped (2440) + Start at 17:48:56 + Duration 194.32s (transform 2.90s, setup 0ms, collect 35.01s, tests 501.26s, environment 18ms, prepare 5.84s) + +EXIT=1 diff --git a/evidence/511-named-gate-diagnostics/mutation-restored.txt b/evidence/511-named-gate-diagnostics/mutation-restored.txt new file mode 100644 index 000000000..eea77cad8 --- /dev/null +++ b/evidence/511-named-gate-diagnostics/mutation-restored.txt @@ -0,0 +1,28 @@ +$ git checkout HEAD -- src/named-gate-lowering.ts # restore +$ git hash-object src/named-gate-lowering.ts # after restore +8a4ac50391d95c95693eb20cdbc69869a5d02060 +$ git rev-parse HEAD:packages/sdk/src/named-gate-lowering.ts +8a4ac50391d95c95693eb20cdbc69869a5d02060 +$ git status --porcelain -- src tests tsconfig.tests.json # empty = worktree identical to HEAD +(status exit 0) +$ git diff --stat HEAD -- . + +$ RELAYFLOWD_BIN=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd npm run build && npx vitest run tests/named-gate-diagnostics.test.ts tests/named-gate-journal.test.ts + +> @relayflows/sdk@2.0.22 build +> tsc && node scripts/make-cli-executable.mjs + + + RUN v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk + + ✓ tests/named-gate-diagnostics.test.ts (17 tests) 685ms + ✓ tests/named-gate-journal.test.ts (5 tests) 1929ms + ✓ a lowered subprocess_gate journals the gate command's streams > persists what the gate printed before a timeout killed it 832ms + ✓ a gate on an agent step > journals the command's streams and surfaces them in the authored failure 793ms + + Test Files 2 passed (2) + Tests 22 passed (22) + Start at 17:41:12 + Duration 3.34s (transform 768ms, setup 0ms, collect 1.65s, tests 2.61s, environment 0ms, prepare 75ms) + +EXIT=0 diff --git a/evidence/511-named-gate-diagnostics/mutation-reverted.txt b/evidence/511-named-gate-diagnostics/mutation-reverted.txt new file mode 100644 index 000000000..5dc6a4e77 --- /dev/null +++ b/evidence/511-named-gate-diagnostics/mutation-reverted.txt @@ -0,0 +1,295 @@ +$ git hash-object src/named-gate-lowering.ts # before revert +8a4ac50391d95c95693eb20cdbc69869a5d02060 +$ git rev-parse HEAD:packages/sdk/src/named-gate-lowering.ts +8a4ac50391d95c95693eb20cdbc69869a5d02060 + +$ git checkout e21caad -- src/named-gate-lowering.ts # revert production file to the parent revision +$ git diff --stat HEAD -- src/named-gate-lowering.ts + packages/sdk/src/named-gate-lowering.ts | 40 +++++---------------------------- + 1 file changed, 5 insertions(+), 35 deletions(-) +$ git hash-object src/named-gate-lowering.ts # after revert +479136c28de4f97c5434800577999bbbb1eca59c +$ git rev-parse e21caad:packages/sdk/src/named-gate-lowering.ts +479136c28de4f97c5434800577999bbbb1eca59c + +$ RELAYFLOWD_BIN=/home/daytona/.relayflows-toolchain/target/2962130851/debug/relayflowd npm run build && npx vitest run tests/named-gate-diagnostics.test.ts tests/named-gate-journal.test.ts + +> @relayflows/sdk@2.0.22 build +> tsc && node scripts/make-cli-executable.mjs + + + RUN v2.1.9 /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk + + ❯ tests/named-gate-diagnostics.test.ts (17 tests | 12 failed) 626ms + × a gate that fails before the command runs says so > names the selection that missed, and does not run the command 33ms + → expected '' to contain 'subprocess_gate' + × a gate that fails before the command runs says so > reports an index into an object and a key into an array as selection failures 23ms + → expected '' to contain 'from_output [0]' + × a gate that fails before the command runs says so > reports a selected value that is not text 34ms + → expected '' to contain 'could not be read as text' + × a gate that fails before the command runs says so > reports a NUL byte in the selected text 23ms + → expected '' to contain 'NUL byte' + × a gate that fails before the command runs says so > keeps the diagnostic to one line so it cannot be mistaken for command output 21ms + → expected '' to contain 'from_output' + × a gate command that never exits normally is not reported as a plain failure > names the signal that killed it 26ms + → expected '' to contain 'SIGKILL' + × a gate command that never exits normally is not reported as a plain failure > names the spawn error and the input size when the child cannot start 26ms + → expected '' to contain 'E2BIG' + × diagnostics never reach the stdout a gate verdict is read from > leaves references_input stdout empty when the selection misses 77ms + → expected '' to contain 'in_output_at ["missing"]' + × diagnostics never reach the stdout a gate verdict is read from > leaves regex_match stdout empty when the selection misses 50ms + → expected '' to contain 'regex_match' + × word_count_bounds reports why its own child failed > reports a nonzero exit with what wc said 45ms + → expected '' to contain 'exited 2' + × word_count_bounds reports why its own child failed > reports output that is not a count 40ms + → expected '' to contain 'not a number' + × word_count_bounds reports why its own child failed > reports a signal rather than an empty exit 1 29ms + → expected '' to contain 'SIGKILL' + ❯ tests/named-gate-journal.test.ts (5 tests | 1 failed) 1808ms + × a lowered subprocess_gate journals the gate command's streams > journals the diagnostic when the gate never reached the command 93ms + → expected '' to contain 'subprocess_gate: from_output ["review…' + ✓ a lowered subprocess_gate journals the gate command's streams > persists what the gate printed before a timeout killed it 823ms + ✓ a gate on an agent step > journals the command's streams and surfaces them in the authored failure 713ms + +⎯⎯⎯⎯⎯⎯ Failed Tests 13 ⎯⎯⎯⎯⎯⎯⎯ + + FAIL tests/named-gate-diagnostics.test.ts > a gate that fails before the command runs says so > names the selection that missed, and does not run the command +AssertionError: expected '' to contain 'subprocess_gate' + +- Expected ++ Received + +- subprocess_gate + + ❯ tests/named-gate-diagnostics.test.ts:117:28 + 115| expect(capture.status).toBe(1); + 116| expect(capture.stdout).not.toContain('THE-COMMAND-RAN'); + 117| expect(capture.stderr).toContain('subprocess_gate'); + | ^ + 118| expect(capture.stderr).toContain('from_output ["review","verdict"]… + 119| expect(capture.stderr).toContain('"verdict"'); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > a gate that fails before the command runs says so > reports an index into an object and a key into an array as selection failures +AssertionError: expected '' to contain 'from_output [0]' + +- Expected ++ Received + +- from_output [0] + + ❯ tests/named-gate-diagnostics.test.ts:124:74 + 122| it('reports an index into an object and a key into an array as selec… + 123| const indexed = gateCommand({ type: 'subprocess_gate', command: se… + 124| expect(runGate(indexed, { output: { '0': 'not an array' } }).stder… + | ^ + 125| + 126| const keyed = gateCommand({ type: 'subprocess_gate', command: sent… + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > a gate that fails before the command runs says so > reports a selected value that is not text +AssertionError: expected '' to contain 'could not be read as text' + +- Expected ++ Received + +- could not be read as text + + ❯ tests/named-gate-diagnostics.test.ts:138:28 + 136| expect(capture.status).toBe(1); + 137| expect(capture.stdout).not.toContain('THE-COMMAND-RAN'); + 138| expect(capture.stderr).toContain('could not be read as text'); + | ^ + 139| }); + 140| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[3/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > a gate that fails before the command runs says so > reports a NUL byte in the selected text +AssertionError: expected '' to contain 'NUL byte' + +- Expected ++ Received + +- NUL byte + + ❯ tests/named-gate-diagnostics.test.ts:147:28 + 145| expect(capture.status).toBe(1); + 146| expect(capture.stdout).not.toContain('THE-COMMAND-RAN'); + 147| expect(capture.stderr).toContain('NUL byte'); + | ^ + 148| }); + 149| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[4/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > a gate that fails before the command runs says so > keeps the diagnostic to one line so it cannot be mistaken for command output +AssertionError: expected '' to contain 'from_output' + +- Expected ++ Received + +- from_output + + ❯ tests/named-gate-diagnostics.test.ts:163:28 + 161| const capture = runGate(command, { output: { other: 1 } }); + 162| + 163| expect(capture.stderr).toContain('from_output'); + | ^ + 164| expect(capture.stderr.trim().split('\n')).toHaveLength(1); + 165| }); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[5/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > a gate command that never exits normally is not reported as a plain failure > names the signal that killed it +AssertionError: expected '' to contain 'SIGKILL' + +- Expected ++ Received + +- SIGKILL + + ❯ tests/named-gate-diagnostics.test.ts:179:28 + 177| // kernel's pipe; buffering them for a post-wait flush would lose … + 178| expect(capture.stdout).toContain('partial'); + 179| expect(capture.stderr).toContain('SIGKILL'); + | ^ + 180| }); + 181| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[6/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > a gate command that never exits normally is not reported as a plain failure > names the spawn error and the input size when the child cannot start +AssertionError: expected '' to contain 'E2BIG' + +- Expected ++ Received + +- E2BIG + + ❯ tests/named-gate-diagnostics.test.ts:208:28 + 206| + 207| expect(capture.status).toBe(1); + 208| expect(capture.stderr).toContain('E2BIG'); + | ^ + 209| expect(capture.stderr).toContain('10 bytes'); + 210| }); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[7/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > diagnostics never reach the stdout a gate verdict is read from > leaves references_input stdout empty when the selection misses +AssertionError: expected '' to contain 'in_output_at ["missing"]' + +- Expected ++ Received + +- in_output_at ["missing"] + + ❯ tests/named-gate-diagnostics.test.ts:220:28 + 218| expect(capture.status).toBe(1); + 219| expect(capture.stdout).toBe(''); + 220| expect(capture.stderr).toContain('in_output_at ["missing"]'); + | ^ + 221| // The receipt `output_contains` looks for must never be forgeable… + 222| // diagnostic that merely names the gate. + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[8/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > diagnostics never reach the stdout a gate verdict is read from > leaves regex_match stdout empty when the selection misses +AssertionError: expected '' to contain 'regex_match' + +- Expected ++ Received + +- regex_match + + ❯ tests/named-gate-diagnostics.test.ts:232:28 + 230| expect(capture.status).toBe(1); + 231| expect(capture.stdout).toBe(''); + 232| expect(capture.stderr).toContain('regex_match'); + | ^ + 233| }); + 234| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[9/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > word_count_bounds reports why its own child failed > reports a nonzero exit with what wc said +AssertionError: expected '' to contain 'exited 2' + +- Expected ++ Received + +- exited 2 + + ❯ tests/named-gate-diagnostics.test.ts:262:28 + 260| expect(capture.status).toBe(1); + 261| expect(capture.stdout).toBe(''); + 262| expect(capture.stderr).toContain('exited 2'); + | ^ + 263| expect(capture.stderr).toContain('wc: read error'); + 264| }); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[10/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > word_count_bounds reports why its own child failed > reports output that is not a count +AssertionError: expected '' to contain 'not a number' + +- Expected ++ Received + +- not a number + + ❯ tests/named-gate-diagnostics.test.ts:272:28 + 270| expect(capture.status).toBe(1); + 271| expect(capture.stdout).toBe(''); + 272| expect(capture.stderr).toContain('not a number'); + | ^ + 273| }); + 274| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[11/13]⎯ + + FAIL tests/named-gate-diagnostics.test.ts > word_count_bounds reports why its own child failed > reports a signal rather than an empty exit 1 +AssertionError: expected '' to contain 'SIGKILL' + +- Expected ++ Received + +- SIGKILL + + ❯ tests/named-gate-diagnostics.test.ts:280:28 + 278| + 279| expect(capture.status).toBe(1); + 280| expect(capture.stderr).toContain('SIGKILL'); + | ^ + 281| }); + 282| }); + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[12/13]⎯ + + FAIL tests/named-gate-journal.test.ts > a lowered subprocess_gate journals the gate command's streams > journals the diagnostic when the gate never reached the command +AssertionError: expected '' to contain 'subprocess_gate: from_output ["review…' + +- Expected ++ Received + +- subprocess_gate: from_output ["review","verdict"] + + ❯ tests/named-gate-journal.test.ts:103:38 + 101| expect(gate.output?.exit_code).toBe(1); + 102| expect(gate.output?.stdout_tail).not.toContain('THE-COMMAND-RAN'); + 103| expect(gate.output?.stderr_tail).toContain('subprocess_gate: from_… + | ^ + 104| }, 30_000); + 105| + +⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[13/13]⎯ + + Test Files 2 failed (2) + Tests 13 failed | 9 passed (22) + Start at 17:40:58 + Duration 3.11s (transform 706ms, setup 0ms, collect 1.51s, tests 2.43s, environment 0ms, prepare 77ms) + +EXIT=1 diff --git a/evidence/511-named-gate-diagnostics/restore-identity.txt b/evidence/511-named-gate-diagnostics/restore-identity.txt new file mode 100644 index 000000000..fe30ca427 --- /dev/null +++ b/evidence/511-named-gate-diagnostics/restore-identity.txt @@ -0,0 +1,17 @@ +# Restoring the working tree to HEAD after the in-place baseline. +$ git checkout HEAD -- packages/sdk +$ git status --porcelain -- packages/sdk # empty = tree and index identical to HEAD +$ git diff --stat HEAD -- packages/sdk +$ for f in src/named-gate-lowering.ts tests/named-gate-diagnostics.test.ts tests/named-gate-journal.test.ts tsconfig.tests.json; do git hash-object packages/sdk/$f; git rev-parse HEAD:packages/sdk/$f; done +# src/named-gate-lowering.ts +8a4ac50391d95c95693eb20cdbc69869a5d02060 +8a4ac50391d95c95693eb20cdbc69869a5d02060 +# tests/named-gate-diagnostics.test.ts +b35c60eb24f577a7e8cee998ddf6fb117e9261c5 +b35c60eb24f577a7e8cee998ddf6fb117e9261c5 +# tests/named-gate-journal.test.ts +2a9f695b88b0f3261fd71bcf5f0b94d6564b1b03 +2a9f695b88b0f3261fd71bcf5f0b94d6564b1b03 +# tsconfig.tests.json +78b1d3d637f77a82013c2d64e4c863d14c44652b +78b1d3d637f77a82013c2d64e4c863d14c44652b diff --git a/evidence/511-named-gate-diagnostics/unrelated-failure-cause.txt b/evidence/511-named-gate-diagnostics/unrelated-failure-cause.txt new file mode 100644 index 000000000..995c43a65 --- /dev/null +++ b/evidence/511-named-gate-diagnostics/unrelated-failure-cause.txt @@ -0,0 +1,13 @@ +# Why the 22 stuck-run-triage failures happen in this sandbox: two installed copies +# of @relayflows/surface are reachable, so the test and the workflow under test get +# different module instances and the WeakMap brand check in getFlowDefinition +# (packages/surface/src/flow.ts:137-143) misses. + +$ node -e "const{createRequire}=require(\"node:module\");for(const from of [\"packages/sdk/tests/x.js\",\"workflows/x.js\"]){const r=createRequire(process.cwd()+\"/\"+from);console.log(from,\"->\",r.resolve(\"@relayflows/surface/runtime\"))}" +packages/sdk/tests/x.js -> /home/daytona/.relayflow-v2-supervisor/durable/repository/packages/sdk/node_modules/@relayflows/surface/dist/runtime.js +workflows/x.js -> /home/daytona/.relayflow-v2-supervisor/durable/node_modules/@relayflows/surface/dist/runtime.js + +$ ls -d /home/daytona/.relayflow-v2-supervisor/durable/node_modules/@relayflows/* # the ancestor install, outside the checkout +/home/daytona/.relayflow-v2-supervisor/durable/node_modules/@relayflows/surface +$ ls -d node_modules 2>&1 # the repository root has no node_modules of its own +ls: cannot access 'node_modules': No such file or directory diff --git a/packages/sdk/src/named-gate-lowering.ts b/packages/sdk/src/named-gate-lowering.ts index 479136c28..8a4ac5039 100644 --- a/packages/sdk/src/named-gate-lowering.ts +++ b/packages/sdk/src/named-gate-lowering.ts @@ -71,19 +71,33 @@ process.exit(artifacts.includes(${JSON.stringify(gate.path)})?0:1);`)}`; } const path = gate.type === 'subprocess_gate' ? gate.from_output : gate.type === 'word_count_bounds' ? undefined : gate.in_output_at; + const pathKey = gate.type === 'subprocess_gate' ? 'from_output' : 'in_output_at'; // Only compiler-owned code is serialized. Author strings are JSON literals; // upstream output travels exclusively through FLOWS_INPUT, never shell text. + // + // `fail` answers a gate that exits 1 having said nothing. Every exit that is + // NOT the author's own predicate or command verdict goes through it: a + // selection that missed, input the child cannot carry, a child that never + // started, a child killed by a signal. It writes ONE bounded line to fd 2 + // and nothing to fd 1 — `references_input` and `word_count_bounds` verify + // against stdout, so a stray byte there would change a verdict. + // + // `writeSync` rather than `process.stderr.write`: on a pipe the latter is + // asynchronous, and the `process.exit` on the same line would drop the + // diagnostic exactly when it is the only account of the failure there is. const setup = `const cp=require('node:child_process'); +const fail=m=>{require('node:fs').writeSync(2,${JSON.stringify(`${gate.type}: `)}+m.replace(/[\\r\\n]+/g,' ').slice(0,400)+'\\n');process.exit(1);}; const input=JSON.parse(process.env.FLOWS_INPUT); let value=input.output; const path=${JSON.stringify(path ?? null)}; if(path!==null){for(const key of path){ if(value===null||typeof value!=='object'||!Object.hasOwn(value,key)|| -(typeof key==='number'?!Array.isArray(value):Array.isArray(value)))process.exit(1); +(typeof key==='number'?!Array.isArray(value):Array.isArray(value))) +fail(${JSON.stringify(`${pathKey} ${JSON.stringify(path ?? null)} selected nothing: no `)}+JSON.stringify(key)+' at that position in the producer output'); value=value[key]; }}else if(${deterministic})value=value.stdout_tail; const text=typeof value==='string'?value:JSON.stringify(value); -if(typeof text!=='string')process.exit(1); +if(typeof text!=='string')fail('the selected value could not be read as text'); `; let body: string; switch (gate.type) { @@ -93,16 +107,32 @@ if(typeof reference!=='string'||reference.length===0||!text.includes(reference)) process.stdout.write('references_input:pass');`; break; case 'subprocess_gate': - body = `if(text.includes('\\0'))process.exit(1); + // `stdio: 'inherit'` hands the child the gate step's own stdout/stderr, + // which ARE the kernel's capture pipes (relayflowd/src/exec_det.rs:78), + // so the command's output is journaled as it is produced — including + // whatever it wrote before a timeout killed the process group. Buffering + // it for a post-wait flush would lose exactly that. What was missing is + // below: `status` is null for every outcome that is not an exit, and all + // of them used to collapse into an indistinguishable bare `exit 1`. + body = `if(text.includes('\\0'))fail('the selected text contains a NUL byte and cannot be passed to the gate command'); const result=cp.spawnSync('/bin/sh',['-c',${JSON.stringify(gate.command)}],{ env:{...process.env,INPUT:text},stdio:'inherit'}); +if(result.error)fail('could not run the gate command: '+(result.error.code||result.error.message)+' (the selected input was '+Buffer.byteLength(text)+' bytes)'); +if(result.signal)fail('the gate command was terminated by '+result.signal); process.exit(result.status===0?0:1);`; break; case 'word_count_bounds': + // The other gate that spawns a child. Its `wc` stderr was piped and then + // discarded, so a broken or absent `wc` was reported as a word count out + // of bounds. A bounded suffix of what it said travels with the cause. body = `const result=cp.spawnSync('wc',['-w'],{input:text,encoding:'utf8',env:{...process.env,LC_ALL:'C'}}); -if(result.status!==0)process.exit(1); +const noise=(result.stderr||'').trim().slice(-200); +const said=noise===''?'':': '+noise; +if(result.error)fail('could not run wc -w: '+(result.error.code||result.error.message)+said); +if(result.signal)fail('wc -w was terminated by '+result.signal+said); +if(result.status!==0)fail('wc -w exited '+result.status+said); const count=result.stdout.trim(); -if(!/^[0-9]+$/.test(count))process.exit(1); +if(!/^[0-9]+$/.test(count))fail('wc -w printed '+JSON.stringify(count.slice(0,80))+' instead of a word count'+said); process.stdout.write(BigInt(count).toString());`; break; case 'regex_match': diff --git a/packages/sdk/tests/named-gate-diagnostics.test.ts b/packages/sdk/tests/named-gate-diagnostics.test.ts new file mode 100644 index 000000000..b35c60eb2 --- /dev/null +++ b/packages/sdk/tests/named-gate-diagnostics.test.ts @@ -0,0 +1,282 @@ +import { spawnSync } from 'node:child_process'; +import { chmodSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join } from 'node:path'; +import { afterEach, describe, expect, it } from 'vitest'; +import { compileSpec } from '../src/compile.js'; +import { lowerNamedGates } from '../src/named-gate-lowering.js'; +import type { NamedDataGate, StepSpec } from '../src/spec.js'; + +/** + * These cases run the REAL lowered command the way the kernel runs it — piped + * stdout and stderr, `FLOWS_INPUT` in the environment (exec_det.rs:72-78) — and + * assert on the captured streams. They establish stream capture and the + * diagnostics; the journal itself is asserted against a live daemon in + * `named-gate-journal.test.ts`, because a captured pipe is not a journal write. + */ + +const directories: string[] = []; +afterEach(() => { for (const directory of directories.splice(0)) rmSync(directory, { recursive: true, force: true }); }); + +function temporaryDirectory(): string { + const directory = mkdtempSync(join(tmpdir(), 'gate-diagnostics-')); + directories.push(directory); + return directory; +} + +/** The command the compiler actually emits for `.gate`. */ +function gateCommand(gate: NamedDataGate, producer: 'deterministic' | 'agent' = 'deterministic'): string { + // `references_input` names a declared input binding, so the producer needs a + // real upstream to bind to; every other gate reads the producer's output. + const referencing = gate.type === 'references_input'; + const source = { + id: 'source', type: 'deterministic', command: 'true', + verification: { type: 'json_schema', schema: { type: 'object', properties: { text: { type: 'string' } } } }, + }; + const step = producer === 'deterministic' + ? { id: 'produce', type: 'deterministic', command: 'true', verification: gate } + : { id: 'produce', type: 'agent', instruction: 'work', cli: 'stub', verification: gate }; + const spec = compileSpec({ + version: '0.1.0', name: 'gated', + steps: (referencing + ? [source, { ...step, input: { reference: { step: 'source', path: ['text'] } } }] + : [step]) as StepSpec[], + }); + const barrier = lowerNamedGates(spec.steps).find(candidate => candidate.id === 'produce.gate'); + const command = (barrier as { command?: string } | undefined)?.command; + if (command === undefined) throw new Error('no gate step was lowered'); + return command; +} + +interface Capture { status: number | null; stdout: string; stderr: string } + +/** + * `input` is the resolved binding the kernel puts in FLOWS_INPUT — the gate's + * own `input` is `{ output: }` (+ `reference` for + * references_input), never the raw envelope. + */ +function runGate(command: string, input: Record, options: { path?: string } = {}): Capture { + const result = spawnSync('/bin/sh', ['-c', command], { + encoding: 'utf8', + env: { + ...process.env, + FLOWS_INPUT: JSON.stringify(input), + ...(options.path === undefined ? {} : { PATH: `${options.path}:${process.env['PATH'] ?? ''}` }), + }, + }); + if (result.error !== undefined) throw result.error; + return { status: result.status, stdout: result.stdout, stderr: result.stderr }; +} + +/** A deterministic producer's journaled envelope. */ +const deterministicEnvelope = (stdout: string) => ({ exit_code: 0, stdout_tail: stdout, stderr_tail: '' }); + +describe('a subprocess_gate keeps the gate command\'s own streams', () => { + const gate = (command: string): NamedDataGate => + ({ type: 'subprocess_gate', command, from_output: ['stdout_tail'] }); + + it('captures stdout and stderr when the command fails', () => { + const command = gateCommand(gate("printf 'GATE_FAILED %s' \"$INPUT\" >&2; printf 'on stdout'; exit 1")); + const capture = runGate(command, { output: deterministicEnvelope('the reviewed text') }); + + expect(capture.status).toBe(1); + expect(capture.stdout).toContain('on stdout'); + expect(capture.stderr).toContain('GATE_FAILED the reviewed text'); + }); + + it('captures stdout and stderr when the command passes', () => { + const command = gateCommand(gate("printf 'GATE_PASSED detail'; printf 'a warning' >&2")); + const capture = runGate(command, { output: deterministicEnvelope('the reviewed text') }); + + expect(capture.status).toBe(0); + expect(capture.stdout).toContain('GATE_PASSED detail'); + expect(capture.stderr).toContain('a warning'); + }); + + it('selects the whole envelope for a non-deterministic producer', () => { + // An agent's envelope has no implicit `stdout_tail` selection, so a gate + // without `from_output` receives the serialized envelope — the shape the + // reported incident's gate was handed. + const command = gateCommand({ type: 'subprocess_gate', command: 'printf %s "$INPUT"; exit 1' }, 'agent'); + const capture = runGate(command, { output: { exit_code: 0, stdout_tail: 'drafted', artifacts: ['a.md'] } }); + + expect(capture.status).toBe(1); + expect(JSON.parse(capture.stdout)).toEqual({ exit_code: 0, stdout_tail: 'drafted', artifacts: ['a.md'] }); + }); +}); + +describe('a gate that fails before the command runs says so', () => { + const sentinel = "printf 'THE-COMMAND-RAN'"; + + it('names the selection that missed, and does not run the command', () => { + const command = gateCommand({ type: 'subprocess_gate', command: sentinel, from_output: ['review', 'verdict'] }); + const capture = runGate(command, { output: { review: { note: 'no verdict here' } } }); + + expect(capture.status).toBe(1); + expect(capture.stdout).not.toContain('THE-COMMAND-RAN'); + expect(capture.stderr).toContain('subprocess_gate'); + expect(capture.stderr).toContain('from_output ["review","verdict"]'); + expect(capture.stderr).toContain('"verdict"'); + }); + + it('reports an index into an object and a key into an array as selection failures', () => { + const indexed = gateCommand({ type: 'subprocess_gate', command: sentinel, from_output: [0] }); + expect(runGate(indexed, { output: { '0': 'not an array' } }).stderr).toContain('from_output [0]'); + + const keyed = gateCommand({ type: 'subprocess_gate', command: sentinel, from_output: ['name'] }); + expect(runGate(keyed, { output: ['an', 'array'] }).stderr).toContain('from_output ["name"]'); + }); + + it('reports a selected value that is not text', () => { + const command = gateCommand({ type: 'subprocess_gate', command: sentinel }); + // A deterministic producer whose envelope has no `stdout_tail` at all: + // `JSON.stringify(undefined)` is `undefined`, not a string. + const capture = runGate(command, { output: { exit_code: 0 } }); + + expect(capture.status).toBe(1); + expect(capture.stdout).not.toContain('THE-COMMAND-RAN'); + expect(capture.stderr).toContain('could not be read as text'); + }); + + it('reports a NUL byte in the selected text', () => { + const command = gateCommand({ type: 'subprocess_gate', command: sentinel, from_output: ['stdout_tail'] }); + const capture = runGate(command, { output: deterministicEnvelope('before\0after') }); + + expect(capture.status).toBe(1); + expect(capture.stdout).not.toContain('THE-COMMAND-RAN'); + expect(capture.stderr).toContain('NUL byte'); + }); + + it('still runs the command for the literal characters backslash and zero', () => { + const command = gateCommand({ type: 'subprocess_gate', command: sentinel, from_output: ['stdout_tail'] }); + const capture = runGate(command, { output: deterministicEnvelope('a literal \\0 sequence') }); + + expect(capture.status).toBe(0); + expect(capture.stdout).toContain('THE-COMMAND-RAN'); + expect(capture.stderr).toBe(''); + }); + + it('keeps the diagnostic to one line so it cannot be mistaken for command output', () => { + const command = gateCommand({ type: 'subprocess_gate', command: sentinel, from_output: ['a\nb'] }); + const capture = runGate(command, { output: { other: 1 } }); + + expect(capture.stderr).toContain('from_output'); + expect(capture.stderr.trim().split('\n')).toHaveLength(1); + }); +}); + +describe('a gate command that never exits normally is not reported as a plain failure', () => { + it('names the signal that killed it', () => { + const command = gateCommand({ + type: 'subprocess_gate', command: 'printf partial; kill -9 $$', from_output: ['stdout_tail'], + }); + const capture = runGate(command, { output: deterministicEnvelope('text') }); + + expect(capture.status).toBe(1); + // `inherit` means the bytes written before the kill are already in the + // kernel's pipe; buffering them for a post-wait flush would lose them. + expect(capture.stdout).toContain('partial'); + expect(capture.stderr).toContain('SIGKILL'); + }); + + it('names the spawn error and the input size when the child cannot start', () => { + // The real generated program, with `node:child_process` stubbed to the + // E2BIG shape (`status: null`, `error.code`) an oversized environment + // produces. A nonexistent command would exit 127 through /bin/sh and + // never reach this branch. + const directory = temporaryDirectory(); + const preload = join(directory, 'stub-spawn.cjs'); + writeFileSync(preload, `const Module = require('node:module'); +const load = Module._load; +Module._load = function (request, ...rest) { + if (request !== 'node:child_process') return load.call(this, request, ...rest); + return { spawnSync: () => ({ + error: Object.assign(new Error('spawnSync /bin/sh E2BIG'), { code: 'E2BIG' }), + status: null, signal: null, stdout: null, stderr: null, + }) }; +}; +`); + writeFileSync(join(directory, 'node'), `#!/bin/sh +exec ${JSON.stringify(process.execPath)} --require ${JSON.stringify(preload)} "$@" +`); + chmodSync(join(directory, 'node'), 0o755); + + const command = gateCommand({ type: 'subprocess_gate', command: 'true', from_output: ['stdout_tail'] }); + const capture = runGate(command, { output: deterministicEnvelope('0123456789') }, { path: directory }); + + expect(capture.status).toBe(1); + expect(capture.stderr).toContain('E2BIG'); + expect(capture.stderr).toContain('10 bytes'); + }); +}); + +describe('diagnostics never reach the stdout a gate verdict is read from', () => { + it('leaves references_input stdout empty when the selection misses', () => { + const command = gateCommand({ type: 'references_input', input_key: 'reference', in_output_at: ['missing'] }); + const capture = runGate(command, { output: { present: 'x' }, reference: 'a required substring' }); + + expect(capture.status).toBe(1); + expect(capture.stdout).toBe(''); + expect(capture.stderr).toContain('in_output_at ["missing"]'); + // The receipt `output_contains` looks for must never be forgeable by a + // diagnostic that merely names the gate. + expect(capture.stdout + capture.stderr).not.toContain('references_input:pass'); + }); + + it('leaves regex_match stdout empty when the selection misses', () => { + const command = gateCommand({ type: 'regex_match', pattern: 'ok', in_output_at: ['missing'] }); + const capture = runGate(command, { output: { present: 'x' } }); + + expect(capture.status).toBe(1); + expect(capture.stdout).toBe(''); + expect(capture.stderr).toContain('regex_match'); + }); + + it('keeps word_count_bounds stdout a bare decimal on success', () => { + const command = gateCommand({ type: 'word_count_bounds', min: 1, max: 5 }); + const capture = runGate(command, { output: deterministicEnvelope('one two three') }); + + expect(capture.status).toBe(0); + expect(capture.stdout).toBe('3'); + expect(capture.stderr).toBe(''); + }); +}); + +describe('word_count_bounds reports why its own child failed', () => { + /** A `wc` earlier on PATH than the real one, behaving as the test needs. */ + function stubWordCount(script: string): string { + const directory = temporaryDirectory(); + writeFileSync(join(directory, 'wc'), `#!/bin/sh\n${script}\n`); + chmodSync(join(directory, 'wc'), 0o755); + return directory; + } + + const command = () => gateCommand({ type: 'word_count_bounds', min: 1, max: 5 }); + + it('reports a nonzero exit with what wc said', () => { + const path = stubWordCount("printf 'wc: read error' >&2; exit 2"); + const capture = runGate(command(), { output: deterministicEnvelope('one two') }, { path }); + + expect(capture.status).toBe(1); + expect(capture.stdout).toBe(''); + expect(capture.stderr).toContain('exited 2'); + expect(capture.stderr).toContain('wc: read error'); + }); + + it('reports output that is not a count', () => { + const path = stubWordCount("printf 'not a number'"); + const capture = runGate(command(), { output: deterministicEnvelope('one two') }, { path }); + + expect(capture.status).toBe(1); + expect(capture.stdout).toBe(''); + expect(capture.stderr).toContain('not a number'); + }); + + it('reports a signal rather than an empty exit 1', () => { + const path = stubWordCount('kill -9 $$'); + const capture = runGate(command(), { output: deterministicEnvelope('one two') }, { path }); + + expect(capture.status).toBe(1); + expect(capture.stderr).toContain('SIGKILL'); + }); +}); diff --git a/packages/sdk/tests/named-gate-journal.test.ts b/packages/sdk/tests/named-gate-journal.test.ts new file mode 100644 index 000000000..2a9f695b8 --- /dev/null +++ b/packages/sdk/tests/named-gate-journal.test.ts @@ -0,0 +1,165 @@ +import { flow } from '@relayflows/surface'; +import { afterEach, describe, expect, it } from 'vitest'; +import { executeAuthoredFlow } from '../src/authored-flow-executor.js'; +import { compileSpec, toKernelSpec } from '../src/compile.js'; +import type { JournalClient } from '../src/journal-client.js'; +import { attachLocalAgent } from '../src/local-agent.js'; +import type { FlowSpec } from '../src/spec.js'; +import { chainFixture } from './flow-chain-fixture.js'; + +/** + * The acceptance the ticket asks for: what a lowered named gate leaves in the + * JOURNAL, read back from a real daemon. `named-gate-diagnostics.test.ts` + * proves the generated program writes the bytes; only these cases prove they + * are persisted as the gate step's `output.stdout_tail` / `output.stderr_tail` + * and survive a failed completion. + */ + +const cleanup: Array<() => Promise> = []; +afterEach(async () => { for (const close of cleanup.splice(0).reverse()) await close(); }); + +interface Envelope { exit_code?: number; stdout_tail?: string; stderr_tail?: string } +interface Completion { completionReason: string; output: Envelope | null; verification?: { verdict?: string } } + +/** The `step.completed` the kernel appended for one step, once it exists. */ +async function completionOf( + journal: JournalClient, runId: string, stepId: string, timeoutMs = 20_000, +): Promise { + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + const { entries } = await journal.journalRead(runId, 1, 1000); + const found = (entries as Array<{ entry_type: string; step_id?: string; payload: Completion }>) + .find(entry => entry.entry_type === 'step.completed' && entry.step_id === stepId); + if (found !== undefined) return found.payload; + await new Promise(resolve => setTimeout(resolve, 25)); + } + throw new Error(`step.completed for "${stepId}" never arrived in ${runId}`); +} + +function gatedSpec(command: string, name: string): FlowSpec { + return compileSpec({ + version: '0.1.0', name, + steps: [{ + id: 'produce', type: 'deterministic', command: "printf 'the reviewed text'", + verification: { type: 'subprocess_gate', command, from_output: ['stdout_tail'] }, + }], + }); +} + +describe('a lowered subprocess_gate journals the gate command\'s streams', () => { + it('keeps both tails on a FAILING gate, the case the operator has nothing else for', async () => { + const fixture = chainFixture(); + cleanup.push(() => fixture.close()); + const journal = await fixture.connect(); + + const started = await journal.runStart(toKernelSpec(gatedSpec( + "printf 'GATE_FAILED tree moved under me'>&2; printf 'GATE_STDOUT %s' \"$INPUT\"; exit 1", + 'gate-fails', + ))); + const gate = await completionOf(journal, started.run_id, 'produce.gate'); + + expect(gate.completionReason).toBe('retries_exhausted'); + expect(gate.output?.exit_code).toBe(1); + expect(gate.output?.stderr_tail).toContain('GATE_FAILED tree moved under me'); + expect(gate.output?.stdout_tail).toContain('GATE_STDOUT the reviewed text'); + }, 30_000); + + it('keeps both tails on a PASSING gate, so a gate that passed for the wrong reason is inspectable', async () => { + const fixture = chainFixture(); + cleanup.push(() => fixture.close()); + const journal = await fixture.connect(); + + const started = await journal.runStart(toKernelSpec(gatedSpec( + "printf 'GATE_PASSED %s' \"$INPUT\"; printf 'a note nobody reads on green'>&2", + 'gate-passes', + ))); + const gate = await completionOf(journal, started.run_id, 'produce.gate'); + + expect(gate.completionReason).toBe('success'); + expect(gate.output?.exit_code).toBe(0); + expect(gate.output?.stdout_tail).toContain('GATE_PASSED the reviewed text'); + expect(gate.output?.stderr_tail).toContain('a note nobody reads on green'); + }, 30_000); + + it('journals the diagnostic when the gate never reached the command', async () => { + const fixture = chainFixture(); + cleanup.push(() => fixture.close()); + const journal = await fixture.connect(); + + const started = await journal.runStart(toKernelSpec(compileSpec({ + version: '0.1.0', name: 'gate-selection-missed', + steps: [{ + id: 'produce', type: 'deterministic', command: "printf 'the reviewed text'", + verification: { + type: 'subprocess_gate', command: "printf 'THE-COMMAND-RAN'", + from_output: ['review', 'verdict'], + }, + }], + }))); + const gate = await completionOf(journal, started.run_id, 'produce.gate'); + + expect(gate.output?.exit_code).toBe(1); + expect(gate.output?.stdout_tail).not.toContain('THE-COMMAND-RAN'); + expect(gate.output?.stderr_tail).toContain('subprocess_gate: from_output ["review","verdict"]'); + }, 30_000); + + it('persists what the gate printed before a timeout killed it', async () => { + const fixture = chainFixture(); + cleanup.push(() => fixture.close()); + const journal = await fixture.connect(); + + // A timeout is not propagated from the producer to the barrier, so it is + // set on the compiled gate step directly. The point is the kernel's: + // `inherit` puts the command's bytes in the capture pipe as they are + // written, so SIGKILL to the process group cannot erase them. + const spec = toKernelSpec(gatedSpec("printf 'GATE_PARTIAL before the wait'; sleep 30", 'gate-times-out')); + const barrier = spec.steps.find(step => step.id === 'produce.gate'); + if (barrier?.type !== 'deterministic') throw new Error('the lowered gate must be a deterministic step'); + barrier.timeout_ms = 750; + + const started = await journal.runStart(spec); + const gate = await completionOf(journal, started.run_id, 'produce.gate'); + + expect(gate.completionReason).toMatch(/timeout|retries_exhausted/u); + expect(gate.output?.stdout_tail).toContain('GATE_PARTIAL before the wait'); + }, 30_000); +}); + +describe('a gate on an agent step', () => { + it('journals the command\'s streams and surfaces them in the authored failure', async () => { + // The reported incident's shape: the gate follows an agent, so the + // envelope it selects from is the worker's, not a deterministic step's. + const fixture = chainFixture(JSON.stringify({ summary: 'drafted the section' })); + cleanup.push(() => fixture.close()); + const journal = await fixture.connect(); + const agent = await attachLocalAgent(journal); + cleanup.push(() => agent.close()); + + const handle = flow('agent-then-gate', async (f) => { + await f.agent('draft', { task: 'Draft the section.' }) + .gate({ + type: 'subprocess_gate', + command: "printf 'GATE_FAILED missing signoff'>&2; printf 'GATE_SAW %s' \"$INPUT\"; exit 1", + }); + f.done('success'); + }); + + const failure = await executeAuthoredFlow(handle, journal, undefined, { + flowPath: fixture.flowPath, localAgentStream: agent.stream, dataDir: fixture.data, + }).then(() => undefined, (error: Error & { runId?: string }) => error); + + expect(failure, 'the failing gate must fail the flow').toBeDefined(); + // The failure report is where an operator meets this first: it must carry + // the command's own account, not just "a gate failed". + expect(failure!.message).toContain('GATE_FAILED missing signoff'); + + const gate = await completionOf(journal, failure!.runId!, 'agent-1.gate'); + expect(gate.output?.exit_code).toBe(1); + expect(gate.output?.stderr_tail).toContain('GATE_FAILED missing signoff'); + // No `from_output`: a non-deterministic producer's envelope is selected + // whole, so the command is handed the serialized envelope rather than a + // `stdout_tail` it does not have. + expect(JSON.parse(gate.output!.stdout_tail!.replace('GATE_SAW ', ''))) + .toEqual({ summary: 'drafted the section' }); + }, 60_000); +}); diff --git a/packages/sdk/tsconfig.tests.json b/packages/sdk/tsconfig.tests.json index 8305cc33a..0adb13b54 100644 --- a/packages/sdk/tsconfig.tests.json +++ b/packages/sdk/tsconfig.tests.json @@ -43,6 +43,8 @@ "tests/direct-input.test.ts", "tests/fixtures/needs-human.flow.ts", "tests/named-gates.test.ts", + "tests/named-gate-diagnostics.test.ts", + "tests/named-gate-journal.test.ts", "tests/build-gate.test.ts", "tests/scope-preflight.test.ts", "tests/worker-cli-cwd.test.ts", diff --git a/summary.md b/summary.md deleted file mode 100644 index e92fc88ed..000000000 --- a/summary.md +++ /dev/null @@ -1,204 +0,0 @@ -# Spend the analysis `flows check` already does - -`flows check` computed the facts that would have prevented five dead runs and -said nothing about them. This spends two of them, and establishes that the -third fact is not a fact. - -## What changed - -### `agent_worker_unresolved` — a check-only warning (permanent) - -A spec with `agent` steps needs a worker *attached for step type `agent`*. -Without one the run parks at the first such step. `flows check` already walks -those steps to print `REQUIRES codex (step "implement"), …` — the analysis -exists; it just stopped one sentence short of the remedy. - -``` -REQUIRES claude (step "implement") -WARNING [agent_worker_unresolved] 2 agent steps ("implement", "review") require an -attached worker. For a local run, use `flows run --local-agent `, unless you -already attach an agent worker for this daemon. `flows check` does not verify -worker attachment: with no worker attached the run parks at the first agent step. -``` - -Design decisions worth naming: - -- **Warning, never a refusal, worded as a requirement rather than a - prediction.** Worker attachment is *unknown* to `flows check`, not absent: - `check` is daemon-free by construction, so it cannot see a worker attached - elsewhere. Saying "this will park" would be a guess. -- **It is a property of the invocation, not of the spec,** so it opts in - through a new `CheckInvocation` seam on `checkFlow` / `checkAuthoredFlow` - rather than entering `preflight`, which stays a pure function of the spec - plus environment probes. Only `flows check` sets it. `flows run` knows the - answer (it attaches its own worker under `--local-agent`), `flows build` and - `flows deploy` check a spec that will run elsewhere, and an SDK caller - reaching `checkAuthoredFlow` directly is generally running a worker already. -- **Scoped to `agent` steps.** `--local-agent` attaches no `llm` worker on the - YAML path, so counting `llm` steps and naming that flag for them would be - false. They are not counted and not mentioned. -- **Counted from the compiled steps, not from `requirements`.** A YAML helper - step (`slack: { post: … }`) compiles to an `agent` step carrying a helper - envelope and needs the same SDK worker, but `requirements` reports it as an - integration, dedupes by harness, and includes `llm` use. Compilation is the - only walk that sees every step that needs the worker. -- **Emitted in the `REQUIRES` position,** because it reads as a footnote to - that line. `emitCheckReport` holds it back from the leading diagnostic batch - and emits it once, after `REQUIRES` and before `CHECK PASSED`; diagnostics - stay on stderr, report lines stay on stdout. `--json` returns before any of - that and keeps the single ordered `diagnostics` array. -- **Emitted alongside a preflight refusal.** An environment refusal is fixed - and rerun; the worker question is still open on the next pass, and staying - silent about it is what made an author meet it one dead run at a time. - -Known blind spot, documented rather than papered over: an authored `.flow.ts` -is checked through `checkMcpHeader`, which preflights a synthetic one-step -header spec without compiling the body, so there are no agent steps to count. -Same blind spot as `permissions_unenforced`. - -### `gate_path_unscanned` — a preflight warning (temporary, retires with #513) - -An `artifact_exists` gate reads the journaled `output.artifacts` list and -nothing else. The bundled worker's *scan* skips any entry whose name starts -with `.` and any entry named exactly `node_modules`, so a gate on a path inside -one of those prefixes cannot rest on the scan, however faithfully the agent -writes the file: - -``` -WARNING [gate_path_unscanned] Step "review" gates on artifact_exists path -".workflow-artifacts/rust/review.md", but the bundled agent worker's artifact scan -records nothing under ".workflow-artifacts": it skips entries whose name starts with -"." and entries named "node_modules". The gate reads the journaled output.artifacts -and never the disk, so writing the file is not enough: the step has to report the -path itself — as object-shaped JSON stdout carrying its own "artifacts" array, as a -completed Relay task output, or from a custom worker. If the gate is meant to rest -on the scan, write the artifact to a path the scan records. -``` - -The issue asked for a refusal, on the premise that such a gate is statically -unsatisfiable. That premise is false, and review F1 is right to reject it: the -scan is only one writer of `output.artifacts`. The same **bundled** worker -promotes object-shaped JSON stdout — and a completed Relay task's output — -verbatim into `output` (`worker.ts`), so an agent that answers with its own -`{"artifacts": [...]}` puts a hidden path in the list and the gate passes; and -`step.complete` accepts any `output` from any worker. Which route a step takes -is a run-time fact, so this reports the scan's limitation and leaves the verdict -to the run. A regression runs the real `AgentWorker` over a fake `claude` that -writes `.workflow-artifacts/review.md` *and* reports it as JSON, then runs the -real lowered gate command over exactly what the worker journaled: the scan -records nothing, the gate exits 0. A refusal would have rejected that spec at -`check`, `run`, `build` and every SDK submission. - -- **One rule, two consumers.** The exclusion predicate moved out of - `agent-artifacts.ts` into `src/artifact-scan-policy.ts`; the real scan now - consumes it, so the warning cannot drift from the scan it describes. A - parity test writes all twelve case paths to a temp dir and asserts the real - `snapshotWorkspaceFiles` output is exactly the predicate's complement. -- **The whole excluded prefix is named,** not the offending segment alone — - that prefix is the directory an author relying on the scan has to move the - artifact out of. -- **Segments are compared exactly, with no normalization.** The gate matches - the author's literal string against the worker's literal list, so - `node_modules-copy/out.md`, `reports/node_modules.md`, `review.md` and - `reports/v1.2/review.md` say nothing, and a backslash is an ordinary - filename character rather than a separator. -- **Collected before preflight's early returns.** `probeNamedGate` runs after - CLI resolution, model governance, scope and budget can each return, so an - author with an unresolved CLI would not learn about the unscanned path until - a later pass — or at run time. A test pins the diagnostic order as - `['gate_path_unscanned', 'cli_unresolved']`. -- **It lives in `PREFLIGHT_WARNING_KINDS`,** so it travels with every public - preflight consumer — `flows check`, `run`, `build` and SDK submissions — and - refuses none of them. - -### `gate_output_not_captured` — **not added**, because it is not true - -The acceptance criterion was conditional: warn that a `subprocess_gate`'s -output is not captured *"for as long as that is true"*. It is not true today. -Verified against a live `relayflowd` before writing anything: - -``` -"stepId": "emit.gate", "completionReason": "retries_exhausted", "exitCode": 1, -"stdoutTail": "GATE_STDOUT_MARKER\n", "stderrTail": "GATE_STDERR_MARKER\n" -``` - -and in the journal itself, on the lowered gate step's `step.completed`: - -```json -"output": { "exit_code": 1, "stdout_tail": "GATE_STDOUT_MARKER\n", - "stderr_tail": "GATE_STDERR_MARKER\n" } -``` - -A warning claiming otherwise would have been a false diagnostic added to a -change whose whole point is that `check` should only say what it knows. What -the criterion actually asks for is that the warning exist exactly while the -bug does — so instead of the warning, this adds a **live-kernel regression -that pins the capture**. If capture ever regresses, that test fails, and the -warning becomes warranted at the moment it becomes true. - -## Tests - -- `tests/artifact-gates.test.ts` — six parameterised warnings asserting the - full excluded prefix is named and that none of them refuses; six lookalike - paths that must say nothing; the scan/predicate parity test against a real - `snapshotWorkspaceFiles` run; the early-return ordering test; an end-to-end - `checkFlow` case pinning the warning and the unrelated refusal beside it; - and the F1 regression described above, which drives the real `AgentWorker` - and the real lowered gate command over an excluded path the agent reports - itself. -- `tests/check-worker-surface.test.ts` (new) — message content, singular vs - plural and the "and N more" truncation, exit code 0, the YAML helper step - being counted where `requirements` omits it, silence for `llm`-only and - deterministic-only flows, survival alongside a preflight refusal, opt-in - discipline, `--json` emitting it once in both the payload and on stderr, and - an **ordered `CliIo` transcript** (both streams in one list) pinning the - position after `REQUIRES` and before `CHECK PASSED` — in both the - `REQUIRES`-present and `REQUIRES`-absent shapes. -- `tests/preflight.test.ts` — `gate_path_unscanned` added to the - warning-reachability list, which asserts set equality against - `PREFLIGHT_WARNING_KINDS`. -- `tests/live-kernel.test.ts` (new block) — the `subprocess_gate` capture - regression, asserting both the rendered message and the journal payload, with - a silent-gate control so the assertions cannot pass on a fixed string. - -`npm test` in `packages/sdk`: 2381 passed, 40 failed. All 40 failures are -pre-existing and environmental — they reproduce identically on a stashed tree -(same 40 failures, same 7 files): `relayflowd` looked up under -`kernel/target/{debug,release}/` while this toolchain builds outside the repo, -and flows needing real harness CLIs. `tsc --noEmit` is clean for `src`, the -type tests, and `tests`. - -## Files - -| File | Change | -| --- | --- | -| `src/artifact-scan-policy.ts` | new — the scan's exclusion rule as a pure predicate, shared | -| `src/named-gate-preflight.ts` | new — the `gate_path_unscanned` warning (retires with #513) | -| `src/cli/check-worker-surface.ts` | new — the `agent_worker_unresolved` warning | -| `src/agent-artifacts.ts` | consumes the extracted predicate instead of its own copy | -| `src/preflight.ts` | collects gate scan coverage before every early return | -| `src/failure-kinds.ts` | the two new kinds, each with its retirement note | -| `src/cli/check.ts` | `CheckInvocation` opt-in seam | -| `src/cli.ts` | opts in at the `check` dispatch; defers the warning to the `REQUIRES` position | -| `docs/SURFACE.md` | documents both kinds where the facts they describe already live | - -Each diagnostic is a distinct kind, so it can be suppressed and later deleted -on its own. Both temporary kinds carry their retirement condition in a comment -at the definition site. - -## Out of scope, unchanged - -The underlying bugs (#511, #513) are not fixed, and the spec is not -round-tripped past the daemon's validator (#502). - -## Merge with origin/main (conflict resolution note) - -Main deliberately broadened the artifact scan so dot-directories like -`.workflow-artifacts/` are journaled and gate-able; only exact names `.git`, -`.relayflowd` and `node_modules` are skipped, at any depth. This branch's -shared `artifact-scan-policy.ts` predicate now encodes THAT set (not the older -dot-prefix rule), so `gate_path_unscanned` warns only on paths the merged scan -can never record — `node_modules/**`, `.git/**`, `.relayflowd/**` — and the -`.workflow-artifacts/` gates main enabled produce no warning. `cli.ts` keeps -both the `agent_worker_unresolved` deferral and main's model-provenance -helper; `agent-artifacts.ts` keeps the shared-predicate import.