Skip to content

Add scuzz diff: behavioral diff of a git revision against the working tree - #165

Merged
SeanCheatham merged 7 commits into
mainfrom
cursor/scuzz-diff-e00b
Sep 24, 2026
Merged

SeanCheatham merged 7 commits into
mainfrom
cursor/scuzz-diff-e00b

Conversation

@SeanCheatham

@SeanCheatham SeanCheatham commented Sep 23, 2026 •

Copy link
Copy Markdown
Owner

scuzz diff [REV] [PATH] compares the behavior of a git revision (default HEAD) with the working tree under TestRuntime.

What it does

  • Delta engine (examples/compiler/src/Delta.scuzz, pure Scuzz): parses timeline v=3 and reports the first diverging state, the number of differing states, length changes, and keyed changes per section (signals by name, a11y/effects/files as multiset lines, token sections by key). It renders the result as text or JSON. Oracles in examples/tyck/delta.scuzz_verify check that a self-diff is empty, that swapping the sides swaps before and after, and that a prefix divergence survives appends.
  • Two-revision builds (examples/compiler/src/Diff.scuzz): REV is checked out in a detached git worktree under build/delta/rev-<sha>. The same compiler builds both sides into build/delta/a and build/delta/b, and each side is cached by fingerprint. The command fails with a clear message when git is missing, the revision is unknown, or the path is not inside a work tree.
  • Paired replay: the workloads are the idle probe, the seeds, and the union of both corpus/ directories, deduplicated by content hash. Each workload is classified as same, diverged, fixed, regressed, or unrunnable. unrunnable covers a missing control or driver, a skipped script event, or schedule drift.
  • Aggregate deltas: added and removed controls, drivers, and oracles; claim verdict changes; Property.sometimes names reached on only one side; shifts in classify counts.
  • Divergence search (--iterations N, --seed): generates inputs over the alphabet both sides share and targets a timeline or pass/fail difference. It shrinks each hit with the existing shrinker, which now takes a predicate (Drive.fuzzShrinkBy), and then shrinks the fault seed and pct_k. Witnesses go to build/delta/witness-<hash>.toml. The command never writes to corpus/.
  • Report: writes build/delta/report.json (v:1, kind:"diff"). Human output is ranked with regressions first. --message-format=json prints the report. The exit code is 1 only when there is a regression.
  • Wiring: adds Cmd.Diff, helpDiff, and a parse fixture (cliDiffArgs). A lone package-directory argument is read as PATH.
  • CI: scripts/ci-delta.sh is a new delta slice, wired into ci.sh (pr, linux-headless) and the fuzz matrix row in ci.yml. It copies examples/counter and examples/shared into a temp git repo and checks these cases:
    • no edit and a whitespace-only edit: all workloads same, exit 0
    • a label change: diverged on a11y, exit 0
    • a broken +1 handler: regressed, and afterPlusShowsOne changes from pass to fail, exit 1
    • --iterations 16: witnesses fail scuzz fuzz --replay on the working tree and pass on the REV worktree, and corpus/ stays unchanged
    • an unknown revision is rejected
  • Docs: manual verify and commands topics (and diff added to the manual command verbs), philosophy.md verification posture, vision.md, and gaps.md Later.

Found along the way

  • Compiler bug (recorded in gaps.md, not fixed): when a lambda is passed directly to a user-defined higher-order def, the compiled lambda body reads the wrong value from a record field of its own parameter. The evaluator is correct. Repro: go(r => IO.pure(Str.len(r.script)), Run("p", "abc")). The first version of the shrinker refactor triggered this bug, and ci-fuzz.sh caught it (the overflow-edge repro stopped shrinking). The shrinker now passes the run to a named def (fuzzShrinkProbe).
  • cliFuzz oracle: it expected a show string without session=false, so it failed on the base branch. The oracle now expects session=false.

Verification

  • CI on this branch: all Linux jobs and macos-smoke pass. linux-fuzz runs the full ci-fuzz.sh and then delta ok (31 s).
  • scuzz check and scuzz fmt --check on compiler, cli, tyck, and manual: no errors
  • scuzz fuzz --iterations 0 on examples/tyck (delta oracles), examples/manual, and a copy of examples/cli: ok
Open in Web Open in Cursor 

cursoragent and others added 7 commits September 23, 2026 23:07
Co-authored-by: sean.cheatham <sean.cheatham@gmail.com>
Co-authored-by: sean.cheatham <sean.cheatham@gmail.com>
…stRuntime.

Co-authored-by: sean.cheatham <sean.cheatham@gmail.com>
Co-authored-by: sean.cheatham <sean.cheatham@gmail.com>
Co-authored-by: sean.cheatham <sean.cheatham@gmail.com>
Co-authored-by: sean.cheatham <sean.cheatham@gmail.com>
…odegen bug.

Co-authored-by: sean.cheatham <sean.cheatham@gmail.com>
@SeanCheatham
SeanCheatham marked this pull request as ready for review September 24, 2026 01:45
@SeanCheatham
SeanCheatham merged commit 8395a08 into main Sep 24, 2026
12 checks passed
@SeanCheatham
SeanCheatham deleted the cursor/scuzz-diff-e00b branch September 24, 2026 01:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants