Skip to content

Remove excluded-file FTS residuals from scoped exports - #5285

Merged
Widthdom merged 2 commits into
mainfrom
fix-issue5274
Sep 6, 2026
Merged

Remove excluded-file FTS residuals from scoped exports#5285
Widthdom merged 2 commits into
mainfrom
fix-issue5274

Conversation

@Widthdom

@Widthdom Widthdom commented Sep 6, 2026

Copy link
Copy Markdown
Owner

Scoped exports could retain excluded-file terms in live FTS5 segments even after deleting their rows and running VACUUM. Rebuild ordinary and available trigram FTS indexes from retained chunks in the private export snapshot before final compaction, hashing, and atomic publication. Legacy databases without trigram support retain their schema; unfiltered full exports keep their existing behavior.

Scoped snapshots also use unpooled SQLite connections, so cancellation and reconstruction failures close the native handle before deleting the private DB, sidecars, and directory. This prevents Windows file-lock failures during cleanup.

Fixes #5274

Validation:

  • After merging origin/main (7fb126a2b), Release export/import and incoming Shell regressions pass: 90 on net8.0 and 90 on net9.0, zero failures/skips. The strengthened interruption theory verifies native-handle closure, complete snapshot cleanup, and destination/source preservation for cancellation and reconstruction exceptions. All four cases failed against the previous implementation on macOS, exposing the same retained-handle cause as Windows CI.
  • The initial local full suite at 1ddf9b2d7 passed: net8.0 12,268 passed / 9 skipped; net9.0 11,779 passed / 435 skipped. This follow-up reran the affected and merged-area tests; Windows validation is delegated to the updated CI run.
  • Shared export fixtures cover every scope selector, empty/all-file scopes, both redaction modes, multiple segments, repeated and previously deleted canaries, raw DB bytes, search, import, source non-mutation, and permissions.
  • The two-file CLI reproduction contains zero excluded-canary bytes; hash/import dry-run validation passes and retained lookup uses indexed_trigram.
  • The issue's repository-file example retains 1 file, 7 chunks, and 18 symbols. Embedded DB size is 663,552 bytes and ZIP size is 113,432 bytes, versus the reported 56,430,592-byte DB and 35,571,238-byte ZIP.
  • Build: zero warnings/errors. Changed-file formatting, git diff --check, changelog validation, and root/workspace index checks pass.
  • Codex adversarial review of origin/main..HEAD: "No blocking/actionable issues found."

Updated the English and Japanese sections of USER_GUIDE.md, DEVELOPER_GUIDE.md, and TESTING_GUIDE.md. Changelog fragment: changelog.d/unreleased/5274.security.md. No CLI flags or JSON/MCP contracts changed. Agent instruction files are unchanged.

Follow-up issues: none discovered within this scope.

@Widthdom
Widthdom merged commit 1d9b64c into main Sep 6, 2026
12 checks passed
@Widthdom
Widthdom deleted the fix-issue5274 branch September 6, 2026 08:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove excluded-file FTS residuals from scoped export archives

1 participant