docs(todo): COMPCOV ffmpeg A/B — null coverage, corpus bloat

[daedalus]

Records the COMPCOV A/B on ffmpeg in docs/TODO.md. Only the TODO text changes; no code.

Setup: stock ffmpeg_read_nosan.so from tools/build_ffmpeg_ready.sh. Its shim is built with cmplog and has 100k const-cmp sites, so both halves of COMPCOV run. Arms are L0, L2, and a second L0 run as a noise control, each over 12 seeds × 2k iterations. Metric: each final corpus is replayed in-process with COMPCOV unset, and the distinct real edge ids are counted.

	L0	L2	L2 vs L0	control vs L0
real edges (median)	13810	13858	5W/7L, p=0.77	3W/9L, p=0.15
corpus entries	714	1215	12W/0L, p=0.0005	8W/4L
eps	6.0	5.8	2W/8L, p=0.11	6W/6L
reported edges (includes marks)	14499	24208	12W/0L	4W/8L

• Coverage: no measurable gain.
• Corpus: L2 keeps 70% more corpus entries for the same coverage (12.1 real edges per entry vs 22.5).
• Throughput: about 5% slower, not significant at n=12.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Qc7F1N9YjmuzLAxiE1XJhp

---

Generated by Claude Code

Summary by Sourcery

Record the ffmpeg COMPCOV A/B results and its coverage, corpus, and throughput implications in the review TODO.

Enhancements:

• Document the ffmpeg COMPCOV A/B results, showing no measurable real-edge coverage gain from Layer 2 and substantially increased corpus size with a modest throughput cost.

Documentation:

• Update the COMPCOV review TODO with the ffmpeg benchmark findings and corpus-efficiency tradeoff.

[sourcery-ai]

Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Updates the COMPCOV review-leftovers TODO entry with a stock-ffmpeg A/B showing no measurable real-coverage benefit, approximately 70% corpus bloat and a nonsignificant ~5% throughput cost for Layer 2, using in-process replay to exclude synthetic marks from the coverage metric.

Flow diagram for the COMPCOV ffmpeg A/B measurement

flowchart LR
    L0[L0 run] --> C0[Final corpus]
    L2[L2 run] --> C2[Final corpus]
    C0 --> R[In-process replay with COMPCOV unset]
    C2 --> R
    R --> M[Count distinct real edge ids]
    M --> O[Compare coverage, corpus size, and throughput]
    CTRL[L0 control run] --> CCTRL[Control corpus]
    CCTRL --> R

Loading

File-Level Changes

Change	Details	Files
Documented a null COMPCOV result on stock ffmpeg while separating real coverage from instrumentation marks.	Added the ffmpeg A/B setup, sample sizes, and replay-based real-edge metric. Recorded that Layer 2 produced no significant real-edge gain and a small, nonsignificant throughput cost. Recorded substantial corpus growth and lower real-edges-per-entry under Layer 2. Preserved the warning that campaign-reported edges include COMPCOV marks and can imply a false win. Updated the remaining stock-target integration gap for making Layer 2 runs reproducible.	docs/TODO.md

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Sorry @daedalus, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 5 days and 11 hours by commenting @sourcery-ai review. Upgrade to get a review now.

[Copilot]

Copilot review overview

🟡 Changes recommended

Correct the contradictory claim, FFmpeg metric values, and corpus-efficiency ratios.

Review effort: Lite
Findings: 2

Open (2)

• Correct ffmpeg metric-trap results and mark-counting values · New
• Narrow claim to stock png_read_tracecmp targets · New

What changed in this PR

Documents FFmpeg COMPCOV A/B results, including coverage, corpus size, and throughput findings.

Changes:

• Adds benchmark measurements to docs/TODO.md.
• Records Layer 2’s corpus growth and lack of measurable coverage gain.

File	Description
docs/​TODO.md	Documents FFmpeg COMPCOV A/B findings.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.