diff --git a/docs/implementation/persistent-review-store.md b/docs/implementation/persistent-review-store.md new file mode 100644 index 0000000..b369566 --- /dev/null +++ b/docs/implementation/persistent-review-store.md @@ -0,0 +1,53 @@ +# Persistent review store (#2) + +This slice follows the library foundation merged at `550340a`. The next approved step is the read-only review screen (#3). It uses Node's built-in SQLite through `runner/store.ts`; no additional runtime dependency is required. + +## Contract and decisions + +Only the trusted runner calls `Store`. Web handlers must send commands through the runner rather than opening this database or passing model/browser claims as ledger entries, approval fingerprints, checkout context, or audited evidence. All production SQL lives in this module. Its database handle is private. + +SQLite owns plan revision allocation. Stable repository/task/plan identity scopes every record; the selected issue is fixed at plan creation. Imports start at revision 1 regardless of the uploaded revision and increment only after validation. Historical revisions and base/head snapshots are append-only. Review writes compare both revision and snapshot ID; imports compare revision. Returned objects are decoded copies. + +Suggestion requests receive opaque UUIDs before an agent response exists. Completion, cancellation, and Apply consult the saved identity, revision, and lifecycle state. Apply loads the saved reply and binding; it atomically saves the next revision, consumes that request, and invalidates pending/ready siblings. Invalid edits roll back without consuming the request. A reply arriving after cancellation or revision change cannot reactivate a request. `getSuggestions` recovers request state and cards after restart. + +Each write transaction takes SQLite's immediate write lock. WAL plus FULL synchronization provides committed recovery; the lock has a five-second busy timeout. Contending processes either serialize or fail explicitly. There is no asynchronous callback inside a transaction. Schema version 1 is installed atomically; unknown versions fail rather than being migrated implicitly. Require Node 26.7.0 or later, matching the CI baseline. + +Ledger entries are immutable within a plan identity. Entries retain full SHA, nullable owner, owned/foreign origin, and immediate source SHA. Rebase records the new base/head, one-to-one mappings, and inherited ownership atomically. A missing or explicitly foreign source yields a foreign destination with null owner. No trailer is consulted. Historical owners survive plan amendments; new normal entries must name a current item. Replaying incompatible ownership is an error, never an upsert. + +Approvals and choices retain the revision and snapshot where the user made the decision. Fingerprints/choice keys from the pure library retain typed file-card object IDs and stable identity. Records are not deleted on unrelated revisions or rebases: the runner recomputes current segments and uses `approvalStates`/`applyChoices` to determine freshness. A stored approval is not itself a claim that the current code is approved. + +Execution checkpoints retain the audited snapshot, revision, executed prefix, actual typed entries, and original out-of-scope paths. Continuation approval is a separate immutable record, requiring a newer revision at the same snapshot. The caller must first reconcile the prefix and validate the remaining suffix. Original scope evidence is never rewritten. Consumers must compare a stored continuation revision with the current revision and checkpoint snapshot before using it. + +## Validation + +`npm test` includes disk-backed SQLite integration tests for revision allocation, cancelled/delayed/cross-plan/replayed suggestions, two independent processes racing Apply, late transaction rollback, abrupt process exit with committed and uncommitted writes, identity isolation, foreign/owned rebase chains, real-Git linking from stored mappings, stale review writes, typed metadata fingerprints, and checkpoint preservation. A child-process startup test treats any SQLite warning as a failure. `npm run typecheck` includes `runner`. + +Baseline: 136 tests. Final counts and CI evidence are recorded in the PR. + +## Remaining integration + +The store does not read Git or execute agents. The runner remains responsible for obtaining immutable typed tree entries and actual filesystem identity; keeping metadata stable; auditing paths, links, and occupancy; reconciling execution prefixes; and phase/container enforcement (#6). Checkpoint persistence is not an implemented execution state machine. This slice adds neither UI nor an HTTP endpoint. No migration against a shared environment is performed. + +Build the review screen under #3 next. The real-issue assignment, experiment protocol, planted-change script, and paired go/no-go experiment remain prerequisites to proceeding beyond that screen. + +## Review round 1 + +Copilot's summary identified two findings (no inline threads). Both were reproduced with failing regressions, then fixed: immutable ledger comparisons now compare the typed fields rather than JSON property order, and a checkpoint's item must be the last item in its completed prefix. No findings were declined. Final suite: 151 tests. + +## Review round 2 + +Reproduced a historical-owner retry failure after an amendment removed the owning item. Normal history writes now apply current-item validation only to new ledger entries; existing SHAs still pass the immutable field comparison. The regression also checks that new commits cannot claim the removed owner and that rebases retain historical ownership. No findings declined. Final suite: 152 tests. + +## Review round 3 + +Reproduced continuation reapproval failing after a second plan amendment. Continuation records now include revision in their immutable primary key; the getter returns the latest approved revision, leaving earlier approvals intact. Repeating approval for the same revision is idempotent, with a separately reproduced regression. This changes the unreleased schema introduced by this PR, not a released database format. + +Declined the duplicate-source-mapping finding as a correctness issue: the rewrites primary key already rejects a repeated source within a snapshot, and the surrounding transaction rolls back its new snapshot and all ledger writes. A new regression passed before any implementation change and confirms the unchanged snapshot, empty ledger, and absence of mappings after rejection. Final suite: 154 tests. + +## Review round 4 + +Reproduced a saved assignment throwing in `applyChoices` after its target item was removed by amendment. Revision commits now discard assignments whose targets no longer exist, in the same transaction as the revision and request invalidation. Unrelated assignments and standalone acceptances remain. Removed assignments cannot reactivate if an ID is later reintroduced. No finding declined this round. Final suite: 155 tests. + +## Review round 5 + +Reproduced and fixed three cases: old approvals reviving after item removal/reintroduction; an unknown identity-mapped SHA remaining unrecorded and later claimable; and the linking engine rejecting historical owners absent from the selected plan revision. Revision commits now discard approvals for removed IDs. Missing identity-map sources receive immutable foreign/null-owner ledger entries. `ownership(identity, revision)` produces a conservative view for that revision, mapping absent owners to null without altering the historical ledger; pass the same revision as the plan supplied to the linking engine. `getLedger` remains the raw provenance record. No findings declined this round. Final suite: 157 tests. diff --git a/runner/store.ts b/runner/store.ts new file mode 100644 index 0000000..f07b609 --- /dev/null +++ b/runner/store.ts @@ -0,0 +1,277 @@ +import type { DatabaseSync, SQLInputValue } from 'node:sqlite'; +import { createRequire } from 'node:module'; +import { randomUUID } from 'node:crypto'; +import { identityKey, type PlanIdentity } from '../core/identity.ts'; +import { importPlan, applySuggestion, assertEditReply, type Plan, type PlanContext, type EditReply } from '../core/plan.ts'; +import type { Approval, SegmentChoice } from '../core/approvals.ts'; + +export function requireSupportedNode(version = process.versions.node): void { + const [major, minor] = version.split('.').map(Number); + if (!major || major < 26 || (major === 26 && (minor ?? 0) < 7)) + throw new Error('codeboost requires Node 26.7.0 or later. Upgrade Node before opening the store.'); +} +export interface Snapshot { id: string; base: string; head: string } +export interface ReviewState { revision: number; snapshotId: string } +export interface LedgerEntry { sha: string; owner: string | null; origin: 'owned' | 'foreign'; sourceSha: string | null } +export interface Checkpoint { + id: string; revision: number; snapshotId: string; item: string; + /** Runner-audited actual tree, retained separately from the declared plan. */ + baseEntries: PlanContext['baseEntries']; completedItems: string[]; outOfScopePaths: string[]; +} +const encode = (value: unknown) => JSON.stringify(value); +const decode = (value: unknown): T => JSON.parse(value as string) as T; +function sha(value: string): void { + if (!/^(?:[a-f0-9]{40}|[a-f0-9]{64})$/.test(value)) throw new Error('Expected a full Git object ID.'); +} + +/** Trusted runner API, not a web/model API. The database handle never escapes this module. + * Callers supply audited Git data and actual checkout path identity; no audit is inferred here. + */ +export class Store { + #db: DatabaseSync; + constructor(path: string) { + requireSupportedNode(); + const { DatabaseSync } = createRequire(import.meta.url)('node:sqlite') as typeof import('node:sqlite'); + this.#db = new DatabaseSync(path, { timeout: 5000 }); + try { + this.#db.exec('PRAGMA foreign_keys=ON; PRAGMA journal_mode=WAL; PRAGMA synchronous=FULL;'); + this.#transaction(() => { + const version = this.#get('PRAGMA user_version')!.user_version; + if (version !== 0 && version !== 1) throw new Error('Unsupported store schema version.'); + if (version === 1) return; + this.#db.exec(` + CREATE TABLE plans (key TEXT PRIMARY KEY, issue INTEGER NOT NULL, revision INTEGER NOT NULL, snapshot_id TEXT); + CREATE TABLE revisions (key TEXT NOT NULL REFERENCES plans(key), revision INTEGER NOT NULL, data TEXT NOT NULL, PRIMARY KEY(key,revision)); + CREATE TABLE snapshots (key TEXT NOT NULL REFERENCES plans(key), id TEXT NOT NULL, data TEXT NOT NULL, PRIMARY KEY(key,id)); + CREATE TABLE requests (id TEXT PRIMARY KEY, key TEXT NOT NULL REFERENCES plans(key), revision INTEGER NOT NULL, state TEXT NOT NULL, reply TEXT); + CREATE TABLE ledger (key TEXT NOT NULL REFERENCES plans(key), sha TEXT NOT NULL, data TEXT NOT NULL, PRIMARY KEY(key,sha)); + CREATE TABLE rewrites (key TEXT NOT NULL REFERENCES plans(key), snapshot_id TEXT NOT NULL, old_sha TEXT NOT NULL, new_sha TEXT NOT NULL, PRIMARY KEY(key,snapshot_id,old_sha)); + CREATE TABLE approvals (key TEXT NOT NULL REFERENCES plans(key), item TEXT NOT NULL, data TEXT NOT NULL, PRIMARY KEY(key,item)); + CREATE TABLE choices (key TEXT NOT NULL REFERENCES plans(key), choice_key TEXT NOT NULL, data TEXT NOT NULL, PRIMARY KEY(key,choice_key)); + CREATE TABLE checkpoints (key TEXT NOT NULL REFERENCES plans(key), id TEXT NOT NULL, data TEXT NOT NULL, PRIMARY KEY(key,id)); + CREATE TABLE continuations (key TEXT NOT NULL REFERENCES plans(key), checkpoint_id TEXT NOT NULL, revision INTEGER NOT NULL, PRIMARY KEY(key,checkpoint_id,revision)); + PRAGMA user_version=1; + `); + }); + } catch (error) { this.#db.close(); throw error; } + } + close(): void { this.#db.close(); } + #get(sql: string, ...args: SQLInputValue[]) { return this.#db.prepare(sql).get(...args); } + #run(sql: string, ...args: SQLInputValue[]) { return this.#db.prepare(sql).run(...args); } + #transaction(fn: () => T): T { + this.#db.exec('BEGIN IMMEDIATE'); + try { const result = fn(); this.#db.exec('COMMIT'); return result; } + catch (error) { this.#db.exec('ROLLBACK'); throw error; } + } + #current(key: string) { + const row = this.#get('SELECT * FROM plans WHERE key=?', key); + if (!row) throw new Error('Unknown plan identity.'); + return row; + } + #expect(key: string, expected: ReviewState): void { + const row = this.#current(key); + if (row.revision !== expected.revision || row.snapshot_id !== expected.snapshotId) throw new Error('Stale review state. Reload before writing.'); + } + #context(key: string, context: PlanContext): void { + if (identityKey(context.identity) !== key || context.issue !== this.#current(key).issue) throw new Error('Plan context identity/issue mismatch.'); + } + #savePlan(key: string, plan: Plan, expected: number): void { + if (this.#run('UPDATE plans SET revision=? WHERE key=? AND revision=?', plan.revision, key, expected).changes !== 1) throw new Error('Stale plan revision.'); + this.#run('INSERT INTO revisions VALUES (?,?,?)', key, plan.revision, encode(plan)); + this.#run("UPDATE requests SET state='invalidated' WHERE key=? AND state IN ('pending','ready')", key); + const items = new Set(plan.items.map(item => item.id)); + for (const row of this.#db.prepare('SELECT item FROM approvals WHERE key=?').all(key)) { + if (!items.has(row.item as string)) this.#run('DELETE FROM approvals WHERE key=? AND item=?', key, row.item!); + } + for (const row of this.#db.prepare('SELECT choice_key,data FROM choices WHERE key=?').all(key)) { + const choice = decode(row.data); + if (choice.action === 'assign' && !items.has(choice.item!)) + this.#run('DELETE FROM choices WHERE key=? AND choice_key=?', key, row.choice_key!); + } + } + createPlan(source: string | Uint8Array, format: 'json' | 'yaml', context: PlanContext, base: string, head: string): Plan { + sha(base); sha(head); + const key = identityKey(context.identity), plan = importPlan(source, format, context, 1).plan; + return this.#transaction(() => { + this.#run('INSERT INTO plans VALUES (?,?,?,NULL)', key, plan.issue, 0); + this.#savePlan(key, plan, 0); + this.#snapshot(key, base, head); + return plan; + }); + } + getPlan(identity: PlanIdentity, revision?: number): Plan { + const key = identityKey(identity), current = this.#current(key); + const row = this.#get('SELECT data FROM revisions WHERE key=? AND revision=?', key, revision ?? current.revision!); + if (!row) throw new Error('Unknown plan revision.'); + return decode(row.data); + } + importRevision(source: string | Uint8Array, format: 'json' | 'yaml', context: PlanContext, expected: number): Plan { + const key = identityKey(context.identity); + return this.#transaction(() => { + this.#context(key, context); + if (this.#current(key).revision !== expected) throw new Error('Stale plan revision.'); + const plan = importPlan(source, format, context, expected + 1).plan; + this.#savePlan(key, plan, expected); return plan; + }); + } + #snapshot(key: string, base: string, head: string): Snapshot { + sha(base); sha(head); + const snapshot = { id: randomUUID(), base, head }; + this.#run('INSERT INTO snapshots VALUES (?,?,?)', key, snapshot.id, encode(snapshot)); + this.#run('UPDATE plans SET snapshot_id=? WHERE key=?', snapshot.id, key); + return snapshot; + } + getSnapshot(identity: PlanIdentity, id?: string): Snapshot { + const key = identityKey(identity); + const row = this.#get('SELECT data FROM snapshots WHERE key=? AND id=?', key, id ?? this.#current(key).snapshot_id!); + if (!row) throw new Error('Unknown snapshot.'); + return decode(row.data); + } + beginSuggestions(identity: PlanIdentity, expectedRevision: number): string { + const key = identityKey(identity); + return this.#transaction(() => { + if (this.#current(key).revision !== expectedRevision) throw new Error('Stale plan revision.'); + const id = randomUUID(); + this.#run("INSERT INTO requests VALUES (?,?,?,'pending',NULL)", id, key, expectedRevision); return id; + }); + } + completeSuggestions(identity: PlanIdentity, id: string, reply: unknown): void { + assertEditReply(reply); + const key = identityKey(identity); + this.#transaction(() => { + const current = this.#current(key); + if (reply.base_revision !== current.revision || this.#run("UPDATE requests SET state='ready',reply=? WHERE id=? AND key=? AND revision=? AND state='pending'", encode(reply), id, key, current.revision!).changes !== 1) + throw new Error('Suggestion request is stale, cancelled, or complete.'); + }); + } + cancelSuggestions(identity: PlanIdentity, id: string): void { + this.#run("UPDATE requests SET state='cancelled' WHERE id=? AND key=? AND state IN ('pending','ready')", id, identityKey(identity)); + } + getSuggestions(identity: PlanIdentity, id: string): { state: string; revision: number; reply: EditReply | null } { + const row = this.#get('SELECT * FROM requests WHERE key=? AND id=?', identityKey(identity), id); + if (!row) throw new Error('Unknown suggestion request.'); + return { state: row.state as string, revision: row.revision as number, reply: row.reply === null ? null : decode(row.reply) }; + } + applySuggestion(identity: PlanIdentity, id: string, index: number, context: PlanContext): Plan { + const key = identityKey(identity); + return this.#transaction(() => { + this.#context(key, context); + const request = this.#get("SELECT * FROM requests WHERE id=? AND key=? AND state='ready'", id, key); + if (!request) throw new Error('Suggestion is unavailable.'); + const plan = this.getPlan(identity); + const next = applySuggestion(plan, decode(request.reply), index, context, { + identity, schemaVersion: plan.schema_version, baseRevision: request.revision as number, issue: plan.issue, + }); + this.#savePlan(key, next, request.revision as number); + this.#run("UPDATE requests SET state='consumed' WHERE id=?", id); return next; + }); + } + #entry(key: string, entry: LedgerEntry): void { + sha(entry.sha); if (entry.sourceSha !== null) sha(entry.sourceSha); + if ((entry.origin === 'foreign' && entry.owner !== null) || (entry.origin === 'owned' && !entry.owner) || !['foreign', 'owned'].includes(entry.origin)) throw new Error('Invalid ledger ownership.'); + const existing = this.#get('SELECT data FROM ledger WHERE key=? AND sha=?', key, entry.sha); + if (existing) { + const prior = decode(existing.data); + if (prior.sha !== entry.sha || prior.owner !== entry.owner || prior.origin !== entry.origin || prior.sourceSha !== entry.sourceSha) + throw new Error('Cannot overwrite immutable ledger ownership.'); + return; + } + this.#run('INSERT INTO ledger VALUES (?,?,?)', key, entry.sha, encode(entry)); + } + /** Trusted runner records commits and updates the observed pair in one transaction. */ + recordHistory(identity: PlanIdentity, expected: ReviewState, base: string, head: string, entries: readonly LedgerEntry[]): Snapshot { + const key = identityKey(identity); + return this.#transaction(() => { + this.#expect(key, expected); + const plan = this.getPlan(identity); + for (const entry of entries) { + const existing = this.#get('SELECT sha FROM ledger WHERE key=? AND sha=?', key, entry.sha); + if (!existing && entry.owner !== null && !plan.items.some(item => item.id === entry.owner)) throw new Error('Unknown ledger owner.'); + this.#entry(key, entry); + } + return this.#snapshot(key, base, head); + }); + } + getLedger(identity: PlanIdentity): LedgerEntry[] { + const key = identityKey(identity); this.#current(key); + return this.#db.prepare('SELECT data FROM ledger WHERE key=? ORDER BY sha').all(key).map(row => decode(row.data)); + } + /** Link a selected revision conservatively; the raw ledger retains historical owners. */ + ownership(identity: PlanIdentity, revision?: number): ReadonlyMap { + const items = new Set(this.getPlan(identity, revision).items.map(item => item.id)); + return new Map(this.getLedger(identity).map(entry => [entry.sha, entry.owner !== null && items.has(entry.owner) ? entry.owner : null])); + } + recordRebase(identity: PlanIdentity, expected: ReviewState, base: string, head: string, mappings: readonly { oldSha: string; newSha: string }[]): Snapshot { + const key = identityKey(identity); + return this.#transaction(() => { + this.#expect(key, expected); + const ledger = new Map(this.getLedger(identity).map(entry => [entry.sha, entry])); + const snapshot = this.#snapshot(key, base, head); + const destinations = new Set(); + for (const { oldSha, newSha } of mappings) { + sha(oldSha); sha(newSha); + if (destinations.has(newSha)) throw new Error('Rebase mappings must be one-to-one.'); + destinations.add(newSha); + const source = ledger.get(oldSha); + if (oldSha !== newSha) this.#entry(key, { sha: newSha, owner: source?.owner ?? null, origin: source?.origin ?? 'foreign', sourceSha: oldSha }); + else if (!source) this.#entry(key, { sha: newSha, owner: null, origin: 'foreign', sourceSha: null }); + this.#run('INSERT INTO rewrites VALUES (?,?,?,?)', key, snapshot.id, oldSha, newSha); + } + return snapshot; + }); + } + getRewrites(identity: PlanIdentity, snapshotId: string): { oldSha: string; newSha: string }[] { + this.getSnapshot(identity, snapshotId); + return this.#db.prepare('SELECT old_sha,new_sha FROM rewrites WHERE key=? AND snapshot_id=? ORDER BY old_sha').all(identityKey(identity), snapshotId).map(row => ({ oldSha: row.old_sha as string, newSha: row.new_sha as string })); + } + /** Values must be computed by the runner from this exact revision/snapshot, never supplied by a browser. */ + saveReview(identity: PlanIdentity, expected: ReviewState, approvals: readonly Approval[], choices: readonly SegmentChoice[]): void { + const key = identityKey(identity); + this.#transaction(() => { + this.#expect(key, expected); const plan = this.getPlan(identity); + for (const approval of approvals) { + if (!plan.items.some(item => item.id === approval.item) || !approval.fingerprint) throw new Error('Invalid approval.'); + this.#run('INSERT OR REPLACE INTO approvals VALUES (?,?,?)', key, approval.item, encode({ ...approval, ...expected })); + } + for (const choice of choices) { + if (!choice.key || !['assign','accept'].includes(choice.action) || (choice.action === 'assign' ? !plan.items.some(item => item.id === choice.item) : choice.item !== null)) throw new Error('Invalid segment choice.'); + this.#run('INSERT OR REPLACE INTO choices VALUES (?,?,?)', key, choice.key, encode({ ...choice, ...expected })); + } + }); + } + getReview(identity: PlanIdentity): { approvals: (Approval & ReviewState)[]; choices: (SegmentChoice & ReviewState)[] } { + const key = identityKey(identity); this.#current(key); + return { + approvals: this.#db.prepare('SELECT data FROM approvals WHERE key=? ORDER BY item').all(key).map(row => decode(row.data)), + choices: this.#db.prepare('SELECT data FROM choices WHERE key=? ORDER BY choice_key').all(key).map(row => decode(row.data)), + }; + } + /** Records evidence from an already completed safety audit; does not authorize execution. */ + recordCheckpoint(identity: PlanIdentity, expected: ReviewState, evidence: Omit): Checkpoint { + const key = identityKey(identity); + return this.#transaction(() => { + this.#expect(key, expected); + const plan = this.getPlan(identity), ids = plan.items.map(item => item.id); + if (!ids.includes(evidence.item) || evidence.completedItems.at(-1) !== evidence.item || new Set(evidence.completedItems).size !== evidence.completedItems.length || evidence.completedItems.some((item, i) => item !== ids[i])) throw new Error('Checkpoint must describe the executed plan prefix.'); + const checkpoint = { ...evidence, ...expected, id: randomUUID() }; + this.#run('INSERT INTO checkpoints VALUES (?,?,?)', key, checkpoint.id, encode(checkpoint)); return checkpoint; + }); + } + getCheckpoint(identity: PlanIdentity, id: string): Checkpoint { + const row = this.#get('SELECT data FROM checkpoints WHERE key=? AND id=?', identityKey(identity), id); + if (!row) throw new Error('Unknown checkpoint.'); return decode(row.data); + } + /** Persist a person's approval only after the runner reconciles the prefix and validates the suffix. */ + approveContinuation(identity: PlanIdentity, checkpointId: string, expected: ReviewState): void { + const key = identityKey(identity); + this.#transaction(() => { + this.#expect(key, expected); const checkpoint = this.getCheckpoint(identity, checkpointId); + if (!checkpoint.outOfScopePaths.length || checkpoint.snapshotId !== expected.snapshotId || expected.revision <= checkpoint.revision) throw new Error('Continuation requires an amended plan at the audited checkpoint.'); + this.#run('INSERT INTO continuations VALUES (?,?,?) ON CONFLICT(key,checkpoint_id,revision) DO NOTHING', key, checkpointId, expected.revision); + }); + } + continuationRevision(identity: PlanIdentity, checkpointId: string): number | null { + this.getCheckpoint(identity, checkpointId); + return (this.#get('SELECT MAX(revision) AS revision FROM continuations WHERE key=? AND checkpoint_id=?', identityKey(identity), checkpointId)?.revision as number | null) ?? null; + } +} diff --git a/test/store.test.ts b/test/store.test.ts new file mode 100644 index 0000000..22bd31b --- /dev/null +++ b/test/store.test.ts @@ -0,0 +1,226 @@ +import { mkdtempSync, rmSync, writeFileSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import { join, resolve } from 'node:path'; +import { execFileSync, spawn } from 'node:child_process'; +import { once } from 'node:events'; +import { afterEach, expect, it } from 'vitest'; +import { Store, requireSupportedNode } from '../runner/store.ts'; +import type { Plan, PlanContext, EditReply } from '../core/plan.ts'; +import { approveItem, approvalStates, choiceKeys, applyChoices } from '../core/approvals.ts'; +import { linkHistory, type Segment } from '../core/linking.ts'; +import { readHistory } from '../git/history.ts'; +const identity = { repositoryId: 'repo', taskId: 'task', planId: 'plan' }; +const context: PlanContext = { identity, issue: 1, baseEntries: [{ path: 'a', kind: 'file' }], pathKey: p => p, allowedCommands: [] }; +const plan = (): Plan => ({ schema_version: 1, revision: 99, issue: 1, summary: 'Example', questions: [], items: [{ id: 'P1', title: 'Change', intent: 'Improve', files: [{ path: 'a', kind: 'edit', renamed_from: null, change: 'Change' }], acceptance: [{ type: 'check', text: 'Works' }], depends_on: [] }] }); +const oid = (n: number) => n.toString(16).padStart(40, '0'); +const reply = (revision = 1): EditReply => ({ schema_version: 1, base_revision: revision, reply: 'Suggestion', edits: [{ op: 'set_field', item: 'P1', summary: 'Rename', reason: 'Clearer', field: 'title', value: 'Updated', file: null, check: null, check_index: null, depends_on: null, new_item: null }] }); +const dirs: string[] = [], stores: Store[] = []; +afterEach(() => { for (const store of stores.splice(0)) store.close(); for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true }); }); +function directory() { const dir = mkdtempSync(join(tmpdir(), 'codeboost-store-')); dirs.push(dir); return dir; } +function open(path: string) { const store = new Store(path); stores.push(store); return store; } +function fixture() { const path = join(directory(), 'state.sqlite'); const store = open(path); store.createPlan(JSON.stringify(plan()), 'json', context, oid(1), oid(2)); return { path, store }; } +const state = (store: Store) => ({ revision: store.getPlan(identity).revision, snapshotId: store.getSnapshot(identity).id }); +function ready(store: Store) { const id = store.beginSuggestions(identity, 1); store.completeSuggestions(identity, id, reply()); return id; } +it('allocates revisions in SQLite, survives reopen, and keeps old revisions and snapshots immutable', () => { + const { store, path } = fixture(); const first = store.getSnapshot(identity); + expect(store.getPlan(identity).revision).toBe(1); + const next = store.importRevision(JSON.stringify({ ...plan(), summary: 'Next' }), 'json', context, 1); + expect(next.revision).toBe(2); expect(store.getPlan(identity, 1).summary).toBe('Example'); + expect(() => store.importRevision(JSON.stringify(plan()), 'json', context, 1)).toThrow(/Stale/); + store.recordHistory(identity, state(store), oid(3), oid(4), []); + const recovered = open(path); + expect(recovered.getPlan(identity)).toEqual(next); expect(recovered.getSnapshot(identity, first.id)).toEqual(first); + expect(recovered.getSnapshot(identity).head).toBe(oid(4)); +}); +it('binds suggestion requests before the reply and rejects cross-plan, cancelled, delayed, replayed, and sibling applications', () => { + const { store } = fixture(); const id = ready(store), sibling = ready(store); + const other = { ...identity, planId: 'other' }; store.createPlan(JSON.stringify(plan()), 'json', { ...context, identity: other }, oid(1), oid(2)); + expect(() => store.applySuggestion(other, id, 0, { ...context, identity: other })).toThrow(/unavailable/); + const cancelled = store.beginSuggestions(identity, 1); store.cancelSuggestions(identity, cancelled); + expect(() => store.completeSuggestions(identity, cancelled, reply())).toThrow(/stale|cancelled/); + const delayed = store.beginSuggestions(identity, 1); + expect(store.applySuggestion(identity, id, 0, context).revision).toBe(2); + expect(() => store.applySuggestion(identity, id, 0, context)).toThrow(/unavailable/); + expect(() => store.applySuggestion(identity, sibling, 0, context)).toThrow(/unavailable/); + expect(() => store.completeSuggestions(identity, delayed, reply())).toThrow(/stale/); +}); +it('rolls back invalid edits without consuming the request or allocating a revision', () => { + const { store } = fixture(); const id = ready(store); + expect(() => store.applySuggestion(identity, id, 3, context)).toThrow(/index/); + expect(store.getPlan(identity).revision).toBe(1); + expect(store.applySuggestion(identity, id, 0, context).items[0]!.title).toBe('Updated'); +}); +it('serializes competing Apply operations across independent processes', async () => { + const { store, path } = fixture(); const ids = [ready(store), ready(store)]; + const source = `import { Store } from ${JSON.stringify(resolve('runner/store.ts'))}; + const store = new Store(process.argv[1]); + process.send('ready'); + process.once('message', () => { try { store.applySuggestion(${JSON.stringify(identity)}, process.argv[2], 0, {...${JSON.stringify(context)}, pathKey: p => p}); process.send('applied'); } + catch { process.send('rejected'); } finally { store.close(); process.disconnect(); } });`; + const children = ids.map(id => spawn(process.execPath, ['--input-type=module', '-e', source, path, id], { stdio: ['ignore', 'pipe', 'pipe', 'ipc'] })); + try { + await Promise.all(children.map(child => once(child, 'message'))); + const outcomes = children.map(child => once(child, 'message')); + const exits = children.map(child => once(child, 'exit')); + children.forEach(child => child.send('apply')); + expect((await Promise.all(outcomes)).map(([result]) => result).sort()).toEqual(['applied', 'rejected']); + await Promise.all(exits); + expect(store.getPlan(identity).revision).toBe(2); + expect(() => store.getPlan(identity, 3)).toThrow(/Unknown/); + } finally { children.forEach(child => child.kill()); } +}, 15000); +it('rolls back the entire ledger batch and snapshot after a late ownership collision', () => { + const { store } = fixture(); store.recordHistory(identity, state(store), oid(1), oid(2), [{ sha: oid(2), owner: 'P1', origin: 'owned', sourceSha: null }]); + const before = store.getSnapshot(identity); + expect(() => store.recordHistory(identity, state(store), oid(1), oid(4), [ + { sha: oid(3), owner: 'P1', origin: 'owned', sourceSha: null }, { sha: oid(2), owner: null, origin: 'foreign', sourceSha: null }, + ])).toThrow(/immutable/); + expect(store.getSnapshot(identity)).toEqual(before); expect(store.getLedger(identity)).toHaveLength(1); + expect(() => store.recordRebase(identity, state(store), oid(5), oid(6), [{ oldSha: oid(2), newSha: oid(6) }, { oldSha: oid(3), newSha: oid(6) }])).toThrow(/one-to-one/); + expect(store.getSnapshot(identity)).toEqual(before); expect(store.getLedger(identity)).toHaveLength(1); +}); +it('preserves owned and missing/null foreign provenance through repeated rebase mappings and restart', () => { + const { store, path } = fixture(); store.recordHistory(identity, state(store), oid(1), oid(3), [ + { sha: oid(2), owner: 'P1', origin: 'owned', sourceSha: null }, { sha: oid(3), owner: null, origin: 'foreign', sourceSha: null }, + ]); + const snapshot = store.recordRebase(identity, state(store), oid(10), oid(14), [2,3,4].map(n => ({ oldSha: oid(n), newSha: oid(n + 10) }))); + store.recordRebase(identity, state(store), oid(20), oid(24), [12,13,14].map(n => ({ oldSha: oid(n), newSha: oid(n + 10) }))); + const recovered = open(path); expect(recovered.ownership(identity).get(oid(22))).toBe('P1'); + expect(recovered.ownership(identity).get(oid(23))).toBeNull(); expect(recovered.ownership(identity).get(oid(24))).toBeNull(); + expect(recovered.getLedger(identity).find(e => e.sha === oid(24))).toEqual({ sha: oid(24), owner: null, origin: 'foreign', sourceSha: oid(14) }); + expect(recovered.getRewrites(identity, snapshot.id)).toHaveLength(3); +}); +it('feeds persisted remapped ownership into the linking engine on real Git history', () => { + const dir = directory(); const git = (...args: string[]) => execFileSync('git', args, { cwd: dir, encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] }).trim(); + git('init', '-b', 'main'); git('config', 'user.name', 'Test'); git('config', 'user.email', 'test@example.invalid'); git('config', 'commit.gpgsign', 'false'); + const commit = () => { git('add', 'a'); git('commit', '-m', 'Change'); return git('rev-parse', 'HEAD'); }; + writeFileSync(join(dir, 'a'), 'before\n'); const base = commit(); writeFileSync(join(dir, 'a'), 'after\n'); const head = commit(); + const store = open(join(directory(), 'state.sqlite')); store.createPlan(JSON.stringify(plan()), 'json', context, base, base); + store.recordHistory(identity, state(store), base, base, [{ sha: oid(9), owner: 'P1', origin: 'owned', sourceSha: null }]); + store.recordRebase(identity, state(store), base, head, [{ oldSha: oid(9), newSha: head }]); + const segments = linkHistory(store.getPlan(identity), readHistory(dir, base, head), store.ownership(identity), p => p); + expect(segments.length).toBeGreaterThan(0); expect(segments.every(s => s.row === 'P1')).toBe(true); + const amended = plan(); amended.items[0]!.id = 'P2'; store.importRevision(JSON.stringify(amended), 'json', context, 1); + const afterRemoval = linkHistory(store.getPlan(identity), readHistory(dir, base, head), store.ownership(identity), p => p); + expect(afterRemoval.every(s => s.row === 'Unplanned')).toBe(true); +}); +it('retains typed file-card approvals/choices while fingerprints detect later metadata and plan changes', () => { + const { store, path } = fixture(); const current = store.getPlan(identity); + const segment: Segment = { path: 'a', oldPath: 'a', kind: 'file', operation: null, content: JSON.stringify({ kind: 'binary', oldObject: { type: 'blob', oid: oid(1) }, newObject: { type: 'blob', oid: oid(2) } }), context: '', owners: ['P1'], row: 'P1', scope: 'in-scope', oldLine: null, newLine: null, hunk: 0, sharesHunkWith: [] }; + const approval = approveItem(current, [segment], 'P1', identity); + const choice = { key: choiceKeys([segment], identity)[0]!, action: 'accept' as const, item: null }; + store.saveReview(identity, state(store), [approval], [choice]); + const recovered = open(path).getReview(identity); + expect(recovered.approvals[0]!.fingerprint).toBe(approval.fingerprint); expect(recovered.choices[0]!.key).toBe(choice.key); + expect(approvalStates(current, [segment], recovered.approvals, identity).P1).toBe('approved'); + expect(approvalStates(current, [{ ...segment, content: segment.content.replace(oid(2), oid(3)) }], recovered.approvals, identity).P1).toBe('stale'); + const old = state(store); store.importRevision(JSON.stringify({ ...plan(), items: [{ ...plan().items[0]!, title: 'Changed' }] }), 'json', context, 1); + expect(() => store.saveReview(identity, old, [approval], [])).toThrow(/Stale/); + expect(approvalStates(store.getPlan(identity), [segment], store.getReview(identity).approvals, identity).P1).toBe('stale'); +}); +it('retains checkpoint scope evidence after amended continuation and rejects a moved head', () => { + const { store, path } = fixture(); const checkpoint = store.recordCheckpoint(identity, state(store), { item: 'P1', completedItems: ['P1'], outOfScopePaths: ['outside'], baseEntries: [...context.baseEntries, { path: 'outside', kind: 'file' }] }); + expect(() => store.approveContinuation(identity, checkpoint.id, state(store))).toThrow(/amended/); + store.importRevision(JSON.stringify(plan()), 'json', context, 1); + store.approveContinuation(identity, checkpoint.id, state(store)); + const recovered = open(path); expect(recovered.getCheckpoint(identity, checkpoint.id)).toEqual(checkpoint); expect(recovered.continuationRevision(identity, checkpoint.id)).toBe(2); + store.recordHistory(identity, state(store), oid(1), oid(3), []); + expect(() => store.approveContinuation(identity, checkpoint.id, state(store))).toThrow(/checkpoint/); +}); +it('rejects unsupported Node versions and opens SQLite without warnings', () => { + expect(() => requireSupportedNode('24.0.0')).toThrow(/Upgrade Node/); expect(() => requireSupportedNode('26.6.0')).toThrow(); + expect(() => requireSupportedNode('26.7.0')).not.toThrow(); + const result = execFileSync(process.execPath, ['--input-type=module', '-e', `import { Store } from './runner/store.ts'; process.on('warning', () => { process.exitCode = 1; }); new Store(':memory:').close();`], { encoding: 'utf8' }); + expect(result).toBe(''); +}); +it('recovers a committed suggestion after abrupt process exit and discards an interrupted transaction', () => { + const path = join(directory(), 'crash.sqlite'); + const source = `import { Store } from './runner/store.ts'; + const store = new Store(process.argv[1]); + const context = {...${JSON.stringify(context)}, pathKey: p => p}; + store.createPlan(${JSON.stringify(JSON.stringify(plan()))}, 'json', context, '${oid(1)}', '${oid(2)}'); + const id = store.beginSuggestions(context.identity, 1); + store.completeSuggestions(context.identity, id, ${JSON.stringify(reply())}); + process.stdout.write(id); process.exit(0);`; + const id = execFileSync(process.execPath, ['--input-type=module', '-e', source, path], { encoding: 'utf8' }); + // Simulate a writer dying between the pointer update and revision insert. + execFileSync(process.execPath, ['--input-type=module', '-e', `import { DatabaseSync } from 'node:sqlite'; const db = new DatabaseSync(process.argv[1]); db.exec('BEGIN IMMEDIATE; UPDATE plans SET revision=2;'); process.exit(0);`, path]); + const recovered = open(path); + expect(recovered.getPlan(identity).revision).toBe(1); + expect(recovered.getSuggestions(identity, id).state).toBe('ready'); + expect(recovered.applySuggestion(identity, id, 0, context).revision).toBe(2); + expect(recovered.getSuggestions(identity, id).state).toBe('consumed'); +}); +it('rolls back an entire review write and refuses approvals tied to an older head', () => { + const { store } = fixture(); const expected = state(store); + const approval = approveItem(store.getPlan(identity), [], 'P1', identity, true); + expect(() => store.saveReview(identity, expected, [approval], [{ key: 'bad', action: 'assign', item: 'P99' }])).toThrow(/choice/); + expect(store.getReview(identity)).toEqual({ approvals: [], choices: [] }); + store.recordHistory(identity, expected, oid(1), oid(5), []); + expect(() => store.saveReview(identity, expected, [approval], [])).toThrow(/Stale/); +}); +it('keeps identical commit SHAs and local item IDs isolated across plan identities', () => { + const { store } = fixture(); + store.recordHistory(identity, state(store), oid(1), oid(2), [{ sha: oid(2), owner: 'P1', origin: 'owned', sourceSha: null }]); + const other = { ...identity, repositoryId: 'other' }; + store.createPlan(JSON.stringify(plan()), 'json', { ...context, identity: other }, oid(1), oid(2)); + expect(store.ownership(other).has(oid(2))).toBe(false); + expect(() => store.getSnapshot(other, store.getSnapshot(identity).id)).toThrow(/Unknown/); +}); +it('accepts idempotent ledger retries regardless of object property order', () => { + const { store } = fixture(); + store.recordHistory(identity, state(store), oid(1), oid(2), [{ sha: oid(2), owner: 'P1', origin: 'owned', sourceSha: null }]); + expect(() => store.recordHistory(identity, state(store), oid(1), oid(2), [{ sourceSha: null, origin: 'owned', owner: 'P1', sha: oid(2) }])).not.toThrow(); + expect(store.getLedger(identity)).toHaveLength(1); +}); +it('requires the checkpoint item to be the last item in the completed prefix', () => { + const { store } = fixture(); const next = plan(); next.items.push({ ...structuredClone(next.items[0]!), id: 'P2' }); + store.importRevision(JSON.stringify(next), 'json', context, 1); + expect(() => store.recordCheckpoint(identity, state(store), { item: 'P1', completedItems: ['P1', 'P2'], outOfScopePaths: ['outside'], baseEntries: context.baseEntries })).toThrow(/prefix/); + expect(store.recordCheckpoint(identity, state(store), { item: 'P2', completedItems: ['P1', 'P2'], outOfScopePaths: ['outside'], baseEntries: context.baseEntries }).item).toBe('P2'); +}); +it('allows historical ledger retries after the owning item is removed, but rejects new entries for it', () => { + const { store } = fixture(); const entry = { sha: oid(2), owner: 'P1', origin: 'owned' as const, sourceSha: null }; + store.recordHistory(identity, state(store), oid(1), oid(2), [entry]); + const next = plan(); next.items[0]!.id = 'P2'; store.importRevision(JSON.stringify(next), 'json', context, 1); + expect(() => store.recordHistory(identity, state(store), oid(1), oid(2), [entry])).not.toThrow(); + expect(() => store.recordHistory(identity, state(store), oid(1), oid(3), [{ ...entry, sha: oid(3) }])).toThrow(/Unknown ledger owner/); + store.recordRebase(identity, state(store), oid(4), oid(5), [{ oldSha: oid(2), newSha: oid(5) }]); + expect(store.getLedger(identity).find(entry => entry.sha === oid(5))!.owner).toBe('P1'); +}); +it('allows a later amended revision to receive a fresh continuation approval', () => { + const { store } = fixture(); const checkpoint = store.recordCheckpoint(identity, state(store), { item: 'P1', completedItems: ['P1'], outOfScopePaths: ['outside'], baseEntries: context.baseEntries }); + store.importRevision(JSON.stringify(plan()), 'json', context, 1); store.approveContinuation(identity, checkpoint.id, state(store)); + expect(() => store.approveContinuation(identity, checkpoint.id, state(store))).not.toThrow(); + store.importRevision(JSON.stringify(plan()), 'json', context, 2); + expect(store.continuationRevision(identity, checkpoint.id)).toBe(2); + expect(() => store.approveContinuation(identity, checkpoint.id, state(store))).not.toThrow(); + expect(store.continuationRevision(identity, checkpoint.id)).toBe(3); +}); +it('rejects duplicate source SHA mappings and rolls back every resulting ledger/snapshot write', () => { + const { store } = fixture(); const before = store.getSnapshot(identity); + expect(() => store.recordRebase(identity, state(store), oid(3), oid(5), [{ oldSha: oid(2), newSha: oid(4) }, { oldSha: oid(2), newSha: oid(5) }])).toThrow(); + expect(store.getSnapshot(identity)).toEqual(before); expect(store.getLedger(identity)).toEqual([]); + expect(store.getRewrites(identity, before.id)).toEqual([]); +}); +it('expires assignments to removed plan items atomically with the amendment', () => { + const { store } = fixture(); + const segment: Segment = { path: 'a', oldPath: 'a', kind: 'text', operation: '+', content: 'after\n', context: '', owners: [null], row: 'Unplanned', scope: 'unplanned', oldLine: null, newLine: 1, hunk: 0, sharesHunkWith: [] }; + store.saveReview(identity, state(store), [], [{ key: choiceKeys([segment], identity)[0]!, action: 'assign', item: 'P1' }]); + const next = plan(); next.items[0]!.id = 'P2'; store.importRevision(JSON.stringify(next), 'json', context, 1); + expect(() => applyChoices(store.getPlan(identity), [segment], store.getReview(identity).choices, identity)).not.toThrow(); + expect(store.getReview(identity).choices).toEqual([]); + store.importRevision(JSON.stringify(plan()), 'json', context, 2); + expect(store.getReview(identity).choices).toEqual([]); +}); +it('does not revive an old approval when a removed item ID is reintroduced', () => { + const { store } = fixture(); store.saveReview(identity, state(store), [approveItem(store.getPlan(identity), [], 'P1', identity, true)], []); + const next = plan(); next.items[0]!.id = 'P2'; store.importRevision(JSON.stringify(next), 'json', context, 1); + store.importRevision(JSON.stringify(plan()), 'json', context, 2); + expect(approvalStates(store.getPlan(identity), [], store.getReview(identity).approvals, identity).P1).toBe('unreviewed'); +}); +it('persists foreign ownership for an unknown SHA mapped to itself', () => { + const { store } = fixture(); store.recordRebase(identity, state(store), oid(1), oid(2), [{ oldSha: oid(2), newSha: oid(2) }]); + expect(store.getLedger(identity)).toEqual([{ sha: oid(2), owner: null, origin: 'foreign', sourceSha: null }]); + expect(() => store.recordHistory(identity, state(store), oid(1), oid(2), [{ sha: oid(2), owner: 'P1', origin: 'owned', sourceSha: null }])).toThrow(/immutable/); +}); diff --git a/tsconfig.json b/tsconfig.json index cfef7d8..0f69925 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -5,5 +5,5 @@ "esModuleInterop": true, "skipLibCheck": true, "noEmit": true, "allowImportingTsExtensions": true, "types": ["node"] }, - "include": ["core/**/*.ts", "git/**/*.ts", "test/**/*.ts"] + "include": ["core/**/*.ts", "git/**/*.ts", "runner/**/*.ts", "test/**/*.ts"] }