diff --git a/AGENTS.md b/AGENTS.md index 022929f93..4b0c8b899 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -112,10 +112,10 @@ Run from repo root through Nix (do not call bare `bun` on the host). Working dir ### Reading SCE Turso databases -- Use the Turso CLI packaged by this repository, not a host `sqlite3`, a generic nixpkgs Turso package, or another Turso version. `nix run .#turso -- --version` must report the version pinned in `flake.nix` (currently `0.7.0`). +- This repository no longer packages a Turso CLI (removed from `flake.nix` in `1b84a0b1`); there is no `nix run .#turso`. Use a Turso shell (`turso` / `tursodb`) whose `--version` matches the `turso` crate version in `cli/Cargo.lock` (currently `0.8.1`). Do not use a host `sqlite3`, nixpkgs `limbo` (older on-disk format), or nixpkgs `turso-cli` (the Turso Cloud CLI, not a database shell). If no matching shell is available, ask the user before inspecting with anything else. - Discover the active repository-scoped Agent Trace DB with `nix run .#sce -- doctor --format json` and read `agent_trace_db.path`. Do not guess the repository ID or inspect legacy `/sce/agent-trace.db` / `agent-trace-.db` files; active storage is `/sce/repos//agent-trace.db`. -- Always enable Turso's multiprocess WAL coordination when opening an SCE database that may also be used by hooks or another agent process: `nix run .#turso -- --experimental-multiprocess-wal --readonly ""`. -- Prefer `--readonly` for inspection. Add a SQL argument for non-interactive reads, for example: `nix run .#turso -- --experimental-multiprocess-wal --readonly "" "PRAGMA quick_check;"`. +- Always enable Turso's multiprocess WAL coordination when opening an SCE database that may also be used by hooks or another agent process: `turso --experimental-multiprocess-wal --readonly ""`. +- Prefer `--readonly` for inspection. Add a SQL argument for non-interactive reads, for example: `turso --experimental-multiprocess-wal --readonly "" "PRAGMA quick_check;"`. Add `-m list` for pipe-separated output that is easier to post-process. - Verify the connection before interpreting data: run `PRAGMA quick_check;` (expect `ok`), `PRAGMA journal_mode;` (expect `wal`), and `SELECT id, applied_at FROM __sce_migrations ORDER BY id;`. - Useful Agent Trace checks include `SELECT COUNT(*) FROM agent_traces;`, `SELECT COUNT(*) FROM messages;`, `SELECT COUNT(*) FROM parts;`, and `SELECT actor_kind, status, COUNT(*) FROM mutation_trace_scopes GROUP BY actor_kind, status;`. - Concurrent inspection processes may temporarily appear as active OpenCode `bash` mutation scopes because the query command itself is a tracked tool call. After the commands finish, `.git/sce/opencode-mutation-scope-state.json` should normally show `"attempts": []` and a clear recovery phase. diff --git a/cli/src/services/doctor/mod.rs b/cli/src/services/doctor/mod.rs index b42906607..8cc796382 100644 --- a/cli/src/services/doctor/mod.rs +++ b/cli/src/services/doctor/mod.rs @@ -1,3 +1,4 @@ +use std::fmt::Write as _; use std::fs; use std::path::{Path, PathBuf}; @@ -11,8 +12,8 @@ use crate::services::lifecycle::{ HealthProblemKind, HealthSeverity, LifecycleProvider, LifecycleProviderId, }; use crate::services::mutation_trace::runtime::{ - inspect_reconciliation_recommendation, run_explicit_reconciliation, ReconciliationFix, - ReconciliationRecommendation, + inspect_reconciliation_recommendation, run_explicit_reconciliation, ReconciliationCounts, + ReconciliationFix, ReconciliationRecommendation, }; use crate::services::output_format::OutputFormat; use crate::services::setup; @@ -214,20 +215,39 @@ async fn execute_doctor_with_lifecycle_providers( } } +fn reconciliation_completed_text(counts: &ReconciliationCounts) -> String { + let mut text = format!( + "Snapshot ref reconciliation completed: deleted {}, retained {}, locally required {}.", + counts.deleted, counts.retained, counts.local_required + ); + if counts.pruned_events > 0 { + let _ = write!( + text, + " Pruned {} mutation event(s) whose snapshot trees are absent from this repository.", + counts.pruned_events + ); + } + if counts.repinned > 0 { + let _ = write!( + text, + " Re-pinned {} snapshot tree(s) that were missing pins.", + counts.repinned + ); + } + text +} + fn reconciliation_fix_result(fix: &ReconciliationFix) -> DoctorFixResultRecord { let (outcome, detail) = match fix { ReconciliationFix::Completed(counts) => ( FixResult::Fixed, - format!( - "Snapshot ref reconciliation completed: deleted {}, retained {}, locally required {}.", - counts.deleted, counts.retained, counts.local_required - ), + reconciliation_completed_text(counts), ), ReconciliationFix::CompletedStatePersistFailed { counts, warning } => ( FixResult::Fixed, format!( - "Snapshot ref reconciliation completed: deleted {}, retained {}, locally required {}. Warning: {warning}", - counts.deleted, counts.retained, counts.local_required + "{} Warning: {warning}", + reconciliation_completed_text(counts) ), ), ReconciliationFix::Failed { @@ -564,6 +584,8 @@ mod reconciliation_fix_tests { deleted: 3, retained: 2, local_required: 1, + repinned: 0, + pruned_events: 0, }; fn not_applied() -> StateWarning { diff --git a/cli/src/services/doctor/render.rs b/cli/src/services/doctor/render.rs index 587eea855..e9555e168 100644 --- a/cli/src/services/doctor/render.rs +++ b/cli/src/services/doctor/render.rs @@ -1028,6 +1028,8 @@ fn reconciliation_counts_json(counts: &ReconciliationCounts) -> serde_json::Valu "deleted": counts.deleted, "retained": counts.retained, "local_required": counts.local_required, + "repinned": counts.repinned, + "pruned_events": counts.pruned_events, }) } @@ -1053,6 +1055,8 @@ mod reconciliation_render_tests { deleted: 3, retained: 2, local_required: 1, + repinned: 0, + pruned_events: 0, }; fn recommendation(last_attempt_age_ms: Option) -> ReconciliationRecommendation { @@ -1087,7 +1091,7 @@ mod reconciliation_render_tests { #[test] fn json_ref_reconciliation_fix_results_match_contract() { - let counts = json!({ "deleted": 3, "retained": 2, "local_required": 1 }); + let counts = json!({ "deleted": 3, "retained": 2, "local_required": 1, "repinned": 0, "pruned_events": 0 }); let cases = [ ( "completed", @@ -1153,7 +1157,7 @@ mod reconciliation_render_tests { #[test] fn json_completed_state_persist_failed_results_match_contract() { - let counts = json!({ "deleted": 3, "retained": 2, "local_required": 1 }); + let counts = json!({ "deleted": 3, "retained": 2, "local_required": 1, "repinned": 0, "pruned_events": 0 }); let cases = [ ( "completed + not applied", diff --git a/cli/src/services/mutation_trace/runtime/coordinator.rs b/cli/src/services/mutation_trace/runtime/coordinator.rs index 1fa06e61d..94eee0341 100644 --- a/cli/src/services/mutation_trace/runtime/coordinator.rs +++ b/cli/src/services/mutation_trace/runtime/coordinator.rs @@ -319,7 +319,7 @@ where fn protected_worktree_failure(error: ProtectedWorktreeError) -> CoordinateError { match error { ProtectedWorktreeError::GitDirResolution(source) - | ProtectedWorktreeError::CheckoutIdentity(source) => CoordinateError::Other(source), + | ProtectedWorktreeError::WorktreeIdentity(source) => CoordinateError::Other(source), ProtectedWorktreeError::LockAcquisition(source) => { CoordinateError::LockAcquisition(anyhow::Error::new(source)) } diff --git a/cli/src/services/mutation_trace/runtime/git_snapshot.rs b/cli/src/services/mutation_trace/runtime/git_snapshot.rs index 0c64a8c9d..ff596bff6 100644 --- a/cli/src/services/mutation_trace/runtime/git_snapshot.rs +++ b/cli/src/services/mutation_trace/runtime/git_snapshot.rs @@ -1,8 +1,10 @@ +use std::collections::BTreeSet; use std::io::Write; use std::path::{Path, PathBuf}; use std::process::Stdio; use anyhow::{anyhow, bail, Context, Result}; +use tokio::io::AsyncWriteExt; use tokio::process::Command; use uuid::Uuid; @@ -102,6 +104,84 @@ impl GitSnapshotService { Ok(()) } + pub async fn pin_trees( + &self, + lease: WorktreeLockLease, + worktree_id: &WorktreeId, + trees: &[TreeId], + ) -> Result<()> { + if trees.is_empty() { + return Ok(()); + } + + let mut stdin_payload = String::new(); + for tree in trees { + stdin_payload.push_str("create "); + stdin_payload.push_str(&pin_ref_name(worktree_id, tree)); + stdin_payload.push(' '); + stdin_payload.push_str(&tree.0); + stdin_payload.push('\n'); + } + + self.run_ref_mutation_inner( + lease, + vec![ + "update-ref".to_string(), + "--no-deref".to_string(), + "--stdin".to_string(), + ], + Some(stdin_payload.into_bytes()), + || {}, + ) + .await?; + Ok(()) + } + + pub async fn absent_trees(&self, trees: &[TreeId]) -> Result> { + if trees.is_empty() { + return Ok(BTreeSet::new()); + } + + let mut stdin_payload = String::new(); + for tree in trees { + stdin_payload.push_str(&tree.0); + stdin_payload.push('\n'); + } + let raw = self + .run_git_with_stdin(&["cat-file", "--batch-check"], stdin_payload.into_bytes()) + .await?; + + let lines: Vec<&str> = raw.lines().collect(); + if lines.len() != trees.len() { + bail!( + "git cat-file --batch-check answered {} line(s) for {} tree(s)", + lines.len(), + trees.len() + ); + } + + let mut absent = BTreeSet::new(); + for (tree, line) in trees.iter().zip(lines) { + let mut fields = line.split(' '); + let object_name = fields.next().unwrap_or_default(); + let object_type = fields.next().unwrap_or_default(); + if object_name != tree.0 { + bail!( + "git cat-file --batch-check answered '{line}' for tree {}", + tree.0 + ); + } + match object_type { + "missing" => { + absent.insert(tree.clone()); + } + "tree" => {} + other => bail!("object {} is a {other}, not a tree", tree.0), + } + } + Ok(absent) + } + pub async fn diff_trees(&self, before: &TreeId, after: &TreeId) -> Result { self.run_git( &[ @@ -336,6 +416,48 @@ impl GitSnapshotService { String::from_utf8(output.stdout) .with_context(|| format!("git {args:?} emitted invalid UTF-8")) } + + async fn run_git_with_stdin(&self, args: &[&str], stdin_payload: Vec) -> Result { + let mut child = self + .cancellable_git_command(args, None) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .with_context(|| { + format!( + "Failed to run git command {:?} in '{}'", + args, + self.repository_root.display() + ) + })?; + + let mut stdin = child + .stdin + .take() + .ok_or_else(|| anyhow!("git {args:?} stdin was not captured"))?; + let writer = tokio::spawn(async move { + stdin.write_all(&stdin_payload).await?; + stdin.shutdown().await + }); + + let output = child + .wait_with_output() + .await + .with_context(|| format!("Failed to wait for git command {args:?}"))?; + writer + .await + .map_err(|source| anyhow!("git {args:?} stdin writer failed: {source}"))? + .with_context(|| format!("Failed to write git command {args:?} stdin"))?; + + if !output.status.success() { + let stderr = String::from_utf8_lossy(&output.stderr).trim().to_string(); + return Err(anyhow!("git {args:?} failed: {stderr}")); + } + + String::from_utf8(output.stdout) + .with_context(|| format!("git {args:?} emitted invalid UTF-8")) + } } #[derive(Clone, Debug, Eq, PartialEq)] diff --git a/cli/src/services/mutation_trace/runtime/protected_worktree.rs b/cli/src/services/mutation_trace/runtime/protected_worktree.rs index eefd57665..50c738360 100644 --- a/cli/src/services/mutation_trace/runtime/protected_worktree.rs +++ b/cli/src/services/mutation_trace/runtime/protected_worktree.rs @@ -27,7 +27,7 @@ pub enum ProtectedWorktreeError { operation: ExternalTaintOperation, source: anyhow::Error, }, - CheckoutIdentity(anyhow::Error), + WorktreeIdentity(anyhow::Error), MarkerWorkerFailed(tokio::task::JoinError), } @@ -46,7 +46,7 @@ impl std::fmt::Display for ProtectedWorktreeError { ), ProtectedWorktreeError::LockAcquisition(source) => write!(f, "{source}"), ProtectedWorktreeError::GitDirResolution(source) - | ProtectedWorktreeError::CheckoutIdentity(source) => write!(f, "{source}"), + | ProtectedWorktreeError::WorktreeIdentity(source) => write!(f, "{source}"), } } } @@ -127,7 +127,7 @@ impl ProtectedWorktree { let worktree_id = resolve_worktree_id(repository_root) .await - .map_err(ProtectedWorktreeError::CheckoutIdentity)?; + .map_err(ProtectedWorktreeError::WorktreeIdentity)?; Ok(Self { marker, diff --git a/cli/src/services/mutation_trace/runtime/ref_doctor.rs b/cli/src/services/mutation_trace/runtime/ref_doctor.rs index 752e224f5..d0a59baa8 100644 --- a/cli/src/services/mutation_trace/runtime/ref_doctor.rs +++ b/cli/src/services/mutation_trace/runtime/ref_doctor.rs @@ -13,6 +13,8 @@ pub(crate) struct ReconciliationCounts { pub(crate) deleted: usize, pub(crate) retained: usize, pub(crate) local_required: usize, + pub(crate) repinned: usize, + pub(crate) pruned_events: usize, } const NOT_APPLIED_MESSAGE: &str = "Maintenance-state update was not applied."; @@ -192,6 +194,8 @@ fn counts(report: ReconciliationReport) -> ReconciliationCounts { deleted: report.deleted, retained: report.retained, local_required: report.local_required, + repinned: report.repinned, + pruned_events: report.pruned_events, } } @@ -199,7 +203,7 @@ fn failure_kind(error: &ReconcileError) -> &'static str { match error { ReconcileError::GitDir(_) => "git_dir_unavailable", ReconcileError::Lock(_) => "lock_unavailable", - ReconcileError::CheckoutIdentity(_) => "worktree_identity_unavailable", + ReconcileError::WorktreeIdentity(_) => "worktree_identity_unavailable", ReconcileError::AgentTraceDbUnavailable(source) => { match source.downcast_ref::() { Some(ExistingRepositoryDbError::Missing { .. }) => "agent_trace_db_missing", @@ -221,6 +225,11 @@ fn failure_kind(error: &ReconcileError) -> &'static str { ReconcileError::MalformedPin { .. } => "malformed_ref", ReconcileError::DurableRoots(_) => "durable_roots_unavailable", ReconcileError::MissingRequiredPins { .. } => "missing_required_pins", + ReconcileError::ObjectInventory(_) => "object_inventory_failed", + ReconcileError::UnrecoverableCursor { .. } => "unrecoverable_cursor", + ReconcileError::PruneTransaction(_) => "prune_transaction_failed", + ReconcileError::PruneConflict => "prune_conflict", + ReconcileError::RepinTransaction(_) => "repin_transaction_failed", ReconcileError::DeleteTransaction(_) => "delete_transaction_failed", } } @@ -305,6 +314,8 @@ mod tests { local_required: 1, retained: 2, deleted: 3, + repinned: 0, + pruned_events: 0, } } @@ -375,6 +386,8 @@ mod tests { local_required: 1, retained: 2, deleted: 3, + repinned: 4, + pruned_events: 5, }; assert_eq!( fix_from_outcome(ExplicitOutcome::Completed(report)), @@ -382,6 +395,8 @@ mod tests { deleted: 3, retained: 2, local_required: 1, + repinned: 4, + pruned_events: 5, }) ); assert_eq!( diff --git a/cli/src/services/mutation_trace/runtime/ref_maintenance.rs b/cli/src/services/mutation_trace/runtime/ref_maintenance.rs index 0c5ce8d92..8c4411815 100644 --- a/cli/src/services/mutation_trace/runtime/ref_maintenance.rs +++ b/cli/src/services/mutation_trace/runtime/ref_maintenance.rs @@ -9,11 +9,10 @@ use super::maintenance_state::{ MaintenanceState, PersistFailure, StateRead, StoredReport, }; use super::ref_reconciliation::{ - reconcile_with_held_lock, ReconcileError, ReconcilePhase, ReconciliationOutcome, + reconcile_with_held_lock, MaintenanceDb, ReconcileError, ReconcilePhase, ReconciliationOutcome, ReconciliationReport, }; use super::worktree_lock::{acquire_inner_async, WorktreeLockError}; -use crate::services::agent_trace_db::repository::RepositoryAgentTraceDb; use crate::services::agent_trace_storage::{ resolve_existing_agent_trace_storage_for_maintenance, AgentTraceStorageContext, }; @@ -72,7 +71,7 @@ pub(super) async fn reconcile_explicit(repository_root: &Path) -> ExplicitOutcom .await } -async fn open_authoritative_db(repository_root: &Path) -> Result { +async fn open_authoritative_db(repository_root: &Path) -> Result { let storage_config = config::resolve_agent_trace_storage_runtime_config(repository_root) .context("Failed to resolve Agent Trace repository storage config.")?; let context = AgentTraceStorageContext { @@ -82,7 +81,10 @@ async fn open_authoritative_db(repository_root: &Path) -> Result( @@ -94,7 +96,7 @@ async fn reconcile_explicit_with( lock_timeout: Duration, ) -> ExplicitOutcome where - P: std::ops::AsyncFnOnce() -> Result, + P: std::ops::AsyncFnOnce() -> Result, C: Fn() -> i64, W: Fn(&Path, &MaintenanceState) -> std::result::Result<(), PersistFailure>, H: FnMut(ReconcilePhase), @@ -201,13 +203,18 @@ fn failure_kind(error: &ReconcileError) -> &'static str { match error { ReconcileError::GitDir(_) => "git_dir", ReconcileError::Lock(_) => "lock", - ReconcileError::CheckoutIdentity(_) => "checkout_identity", + ReconcileError::WorktreeIdentity(_) => "worktree_identity", ReconcileError::AgentTraceDbUnavailable(_) => "agent_trace_db_unavailable", ReconcileError::SnapshotService(_) => "snapshot_service", ReconcileError::PinInventory(_) => "pin_inventory", ReconcileError::MalformedPin { .. } => "malformed_pin", ReconcileError::DurableRoots(_) => "durable_roots", ReconcileError::MissingRequiredPins { .. } => "missing_required_pins", + ReconcileError::ObjectInventory(_) => "object_inventory", + ReconcileError::UnrecoverableCursor { .. } => "unrecoverable_cursor", + ReconcileError::PruneTransaction(_) => "prune_transaction", + ReconcileError::PruneConflict => "prune_conflict", + ReconcileError::RepinTransaction(_) => "repin_transaction", ReconcileError::DeleteTransaction(_) => "delete_transaction", } } @@ -216,7 +223,7 @@ fn failure_kind(error: &ReconcileError) -> &'static str { async fn open_authoritative_db_at_state_root( repository_root: &Path, state_root: &Path, -) -> Result { +) -> Result { let context = AgentTraceStorageContext { repository_root, explicit_repository_id: None, @@ -227,5 +234,8 @@ async fn open_authoritative_db_at_state_root( state_root, ) .await - .map(|storage| storage.db) + .map(|storage| MaintenanceDb { + db: storage.db, + path: storage.db_path, + }) } diff --git a/cli/src/services/mutation_trace/runtime/ref_maintenance/tests.rs b/cli/src/services/mutation_trace/runtime/ref_maintenance/tests.rs index d19be1183..2a7c5a6eb 100644 --- a/cli/src/services/mutation_trace/runtime/ref_maintenance/tests.rs +++ b/cli/src/services/mutation_trace/runtime/ref_maintenance/tests.rs @@ -26,6 +26,8 @@ use crate::services::repository_identity::resolve::resolve_repository_identity; const NOW: i64 = 1_000_000_000_000; const OTHER_REPOSITORY_ID: &str = "some-other-repository"; +const ABSENT_TREE: &str = "1111111111111111111111111111111111111111"; +const ABSENT_TREE_2: &str = "2222222222222222222222222222222222222222"; mod coordination; mod cross_process; @@ -185,6 +187,47 @@ async fn seed_durable_root_for(db_path: &Path, worktree: &str, tree: &str) { .expect("seed durable root"); } +async fn seed_event(db_path: &Path, revision: u64, before_tree: &str, after_tree: &str) { + let db = RepositoryAgentTraceDb::open_without_migrations_at(db_path) + .await + .expect("writer"); + let revision_blob = revision.to_be_bytes(); + db.execute( + "INSERT INTO mutation_trace_events + (worktree_id, revision, before_tree, after_tree, tainted, failure_kind, + attribution_kind, attribution_scope_id, boundary_kind, boundary_scope_id, + boundary_event_id) + VALUES ('main', ?1, ?2, ?3, 0, 'healthy', 'ineligible_unscoped', NULL, 'flush', + NULL, NULL)", + (revision_blob.as_slice(), before_tree, after_tree), + ) + .await + .expect("seed event"); + db.execute( + "INSERT INTO mutation_trace_event_active_scopes (worktree_id, revision, scope_id) + VALUES ('main', ?1, 'scope')", + (revision_blob.as_slice(),), + ) + .await + .expect("seed event active scope"); +} + +async fn event_revisions(db_path: &Path) -> Vec { + let db = RepositoryAgentTraceDb::open_without_migrations_at(db_path) + .await + .expect("reader"); + db.query_map( + "SELECT revision FROM mutation_trace_events WHERE worktree_id = 'main' ORDER BY revision", + (), + |row| { + let blob: Vec = row.get(0)?; + Ok(u64::from_be_bytes(blob.as_slice().try_into()?)) + }, + ) + .await + .expect("event revisions") +} + fn unavailable_cause(error: &ReconcileError) -> &ExistingRepositoryDbError { let ReconcileError::AgentTraceDbUnavailable(source) = error else { panic!("expected AgentTraceDbUnavailable, got {error:?}"); @@ -227,6 +270,8 @@ async fn ref_reconciliation_verified_database_reconciles_orphan_pin_and_records_ local_required: 0, retained: 0, deleted: 1, + repinned: 0, + pruned_events: 0, } ); assert_eq!( @@ -391,7 +436,7 @@ async fn ref_reconciliation_failure_streak_counts_but_lock_contention_never_does let mut phases = Vec::new(); let outcome = reconcile_explicit_with( &fixture.root, - async || -> anyhow::Result { unreachable!("db must not open") }, + async || -> anyhow::Result<_> { unreachable!("db must not open") }, || NOW, |_, _| unreachable!("state must not be written on contention"), |phase| phases.push(phase), @@ -450,14 +495,10 @@ async fn ref_reconciliation_missing_database_and_failed_persistence_keep_both_er } #[tokio::test(flavor = "multi_thread")] -async fn ref_reconciliation_missing_required_pin_and_failed_persistence_keep_both_errors() { +async fn ref_reconciliation_unrecoverable_cursor_and_failed_persistence_keep_both_errors() { let fixture = Fixture::new("https://example.invalid/org/repo-a.git"); fixture.create_db().await; - let snapshot = GitSnapshotService::new(&fixture.root) - .await - .expect("snapshot"); - let unpinned = snapshot.capture_tree().await.expect("tree"); - seed_durable_root(&fixture.db_path(), &unpinned.0).await; + seed_durable_root(&fixture.db_path(), ABSENT_TREE).await; let (outcome, phases) = fixture .run_with(failing_write(PersistPhase::WriteStaging)) @@ -470,10 +511,10 @@ async fn ref_reconciliation_missing_required_pin_and_failed_persistence_keep_bot else { panic!("expected failure, got {outcome:?}"); }; - let ReconcileError::MissingRequiredPins { missing } = &error else { - panic!("expected missing required pins, got {error:?}"); + let ReconcileError::UnrecoverableCursor { tree } = &error else { + panic!("expected unrecoverable cursor, got {error:?}"); }; - assert_eq!(missing, &vec![unpinned]); + assert_eq!(tree.0, ABSENT_TREE); assert!(matches!( state_warning, Some(StatePersistWarning::WriteFailed( @@ -881,6 +922,8 @@ async fn ref_reconciliation_retains_every_repository_wide_durable_root_and_never local_required: 1, retained: 2, deleted: 2, + repinned: 0, + pruned_events: 0, } ); let mut remaining: Vec = fixture diff --git a/cli/src/services/mutation_trace/runtime/ref_maintenance/tests/coordination.rs b/cli/src/services/mutation_trace/runtime/ref_maintenance/tests/coordination.rs index e95d5c313..4bef6d556 100644 --- a/cli/src/services/mutation_trace/runtime/ref_maintenance/tests/coordination.rs +++ b/cli/src/services/mutation_trace/runtime/ref_maintenance/tests/coordination.rs @@ -119,6 +119,8 @@ async fn assert_coordinate_blocked_until_pass_releases(phase: ReconcilePhase) { local_required: 1, retained: 1, deleted: 1, + repinned: 0, + pruned_events: 0, } ); let coordinated = coordinated.expect("the coordinator completes once the pass releases"); diff --git a/cli/src/services/mutation_trace/runtime/ref_maintenance/tests/fail_closed.rs b/cli/src/services/mutation_trace/runtime/ref_maintenance/tests/fail_closed.rs index d8183fb6c..beac351a8 100644 --- a/cli/src/services/mutation_trace/runtime/ref_maintenance/tests/fail_closed.rs +++ b/cli/src/services/mutation_trace/runtime/ref_maintenance/tests/fail_closed.rs @@ -7,12 +7,15 @@ use super::super::super::ref_reconciliation::{ReconcileError, ReconcilePhase}; use super::super::reconcile_explicit_with; use super::super::{ExplicitOutcome, SkipReason}; use super::support::{await_signal, pinned_trees, spawn_parked_pass, write_file}; -use super::{pin_orphan, seed_durable_root, Fixture, NOW}; +use super::{ + event_revisions, pin_orphan, seed_durable_root, seed_event, Fixture, ABSENT_TREE, + ABSENT_TREE_2, NOW, +}; const REMOTE: &str = "https://example.invalid/org/repo-a.git"; #[tokio::test(flavor = "multi_thread")] -async fn ref_reconciliation_missing_local_pin_fails_closed_while_other_orphans_exist() { +async fn ref_reconciliation_repins_missing_local_pin_whose_tree_exists_and_deletes_orphans() { let fixture = Fixture::new(REMOTE); fixture.create_db().await; write_file(&fixture.root, "orphan.txt", "orphan\n"); @@ -23,22 +26,81 @@ async fn ref_reconciliation_missing_local_pin_fails_closed_while_other_orphans_e .expect("snapshot"); let unpinned = snapshot.capture_tree().await.expect("tree"); seed_durable_root(&fixture.db_path(), &unpinned.0).await; - let refs_before = pinned_trees(&fixture.root, "main"); - assert_eq!(refs_before, [orphan].into_iter().collect()); + assert_eq!( + pinned_trees(&fixture.root, "main"), + [orphan].into_iter().collect() + ); + + let (outcome, _) = fixture.run().await; + + let ExplicitOutcome::Completed(report) = outcome else { + panic!("expected completed pass, got {outcome:?}"); + }; + assert_eq!(report.repinned, 1); + assert_eq!(report.pruned_events, 0); + assert_eq!(report.deleted, 1); + assert_eq!( + pinned_trees(&fixture.root, "main"), + [unpinned.0].into_iter().collect() + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn ref_reconciliation_prunes_events_whose_trees_are_absent_and_repins_the_rest() { + let fixture = Fixture::new(REMOTE); + fixture.create_db().await; + let snapshot = super::super::super::git_snapshot::GitSnapshotService::new(&fixture.root) + .await + .expect("snapshot"); + let first = snapshot.capture_tree().await.expect("tree"); + write_file(&fixture.root, "later.txt", "later\n"); + let second = snapshot.capture_tree().await.expect("tree"); + seed_durable_root(&fixture.db_path(), &second.0).await; + seed_event(&fixture.db_path(), 1, ABSENT_TREE, ABSENT_TREE_2).await; + seed_event(&fixture.db_path(), 2, ABSENT_TREE_2, &first.0).await; + seed_event(&fixture.db_path(), 3, &first.0, &second.0).await; + + let (outcome, _) = fixture.run().await; + + let ExplicitOutcome::Completed(report) = outcome else { + panic!("expected completed pass, got {outcome:?}"); + }; + assert_eq!(report.pruned_events, 2); + assert_eq!(report.repinned, 2); + assert_eq!(report.deleted, 0); + assert_eq!(event_revisions(&fixture.db_path()).await, vec![3]); + assert_eq!( + pinned_trees(&fixture.root, "main"), + [first.0, second.0].into_iter().collect() + ); +} + +#[tokio::test(flavor = "multi_thread")] +async fn ref_reconciliation_absent_cursor_tree_fails_closed_without_pruning() { + let fixture = Fixture::new(REMOTE); + fixture.create_db().await; + write_file(&fixture.root, "orphan.txt", "orphan\n"); + let orphan = pin_orphan(&fixture.root).await; + seed_durable_root(&fixture.db_path(), ABSENT_TREE).await; + seed_event(&fixture.db_path(), 1, ABSENT_TREE_2, ABSENT_TREE).await; let (outcome, _) = fixture.run().await; let ExplicitOutcome::Failed { error, .. } = outcome else { panic!("expected failure, got {outcome:?}"); }; - let ReconcileError::MissingRequiredPins { missing } = error else { - panic!("expected missing required pins, got {error:?}"); + assert!(error + .to_string() + .contains("Let one agent tool run in this checkout")); + assert!(error.to_string().contains("rerun `sce doctor --fix`")); + let ReconcileError::UnrecoverableCursor { tree } = error else { + panic!("expected unrecoverable cursor, got {error:?}"); }; - assert_eq!(missing, vec![unpinned]); + assert_eq!(tree.0, ABSENT_TREE); + assert_eq!(event_revisions(&fixture.db_path()).await, vec![1]); assert_eq!( pinned_trees(&fixture.root, "main"), - refs_before, - "the eligible orphan is not deleted when a required pin is missing" + [orphan].into_iter().collect() ); } diff --git a/cli/src/services/mutation_trace/runtime/ref_maintenance/tests/state_recovery.rs b/cli/src/services/mutation_trace/runtime/ref_maintenance/tests/state_recovery.rs index 9a8e0fb6f..fb21d0b78 100644 --- a/cli/src/services/mutation_trace/runtime/ref_maintenance/tests/state_recovery.rs +++ b/cli/src/services/mutation_trace/runtime/ref_maintenance/tests/state_recovery.rs @@ -169,7 +169,7 @@ async fn ref_reconciliation_busy_explicit_pass_leaves_invalid_state_byte_identic let outcome = reconcile_explicit_with( &fixture.root, - async || -> anyhow::Result { unreachable!("db must not open") }, + async || -> anyhow::Result<_> { unreachable!("db must not open") }, || NOW, |_, _| unreachable!("state must not be written"), |_| {}, diff --git a/cli/src/services/mutation_trace/runtime/ref_maintenance/tests/worktrees.rs b/cli/src/services/mutation_trace/runtime/ref_maintenance/tests/worktrees.rs index 70064c806..48c0b52ca 100644 --- a/cli/src/services/mutation_trace/runtime/ref_maintenance/tests/worktrees.rs +++ b/cli/src/services/mutation_trace/runtime/ref_maintenance/tests/worktrees.rs @@ -2,9 +2,9 @@ use std::collections::BTreeSet; use std::path::{Path, PathBuf}; use std::time::Duration; -use super::super::super::git_snapshot::{resolve_worktree_id, GitSnapshotService}; +use super::super::super::git_snapshot::resolve_worktree_id; use super::super::super::maintenance_state::write_state_atomically; -use super::super::super::ref_reconciliation::{ReconcileError, ReconciliationReport}; +use super::super::super::ref_reconciliation::ReconciliationReport; use super::super::{open_authoritative_db_at_state_root, reconcile_explicit_with, ExplicitOutcome}; use super::support::{ coordinate_flush, hook_ok, pinned_trees, sce_refs, start_payload, write_file, writer, @@ -134,6 +134,8 @@ async fn ref_reconciliation_historical_roots_of_a_real_coordinator_sequence_stay local_required: 4, retained: 4, deleted: 1, + repinned: 0, + pruned_events: 0, } ); assert_eq!( @@ -173,6 +175,8 @@ async fn ref_reconciliation_pin_of_one_real_worktree_is_retained_while_another_n local_required: 0, retained: 1, deleted: 1, + repinned: 0, + pruned_events: 0, } ); assert_eq!( @@ -184,7 +188,8 @@ async fn ref_reconciliation_pin_of_one_real_worktree_is_retained_while_another_n } #[tokio::test(flavor = "multi_thread")] -async fn ref_reconciliation_missing_pin_in_another_real_worktree_neither_fails_nor_is_repaired() { +async fn ref_reconciliation_missing_pin_in_another_real_worktree_is_repaired_only_by_its_own_pass() +{ let linked = Linked::new().await; let fixture = &linked.fixture; let db_path = fixture.db_path(); @@ -224,6 +229,8 @@ async fn ref_reconciliation_missing_pin_in_another_real_worktree_neither_fails_n local_required: 0, retained: 1, deleted: 1, + repinned: 0, + pruned_events: 0, } ); assert!(!linked.main_pins().contains(&orphan)); @@ -239,25 +246,15 @@ async fn ref_reconciliation_missing_pin_in_another_real_worktree_neither_fails_n assert_eq!(refs_of_linked_before, refs_of_linked_after); assert_eq!( linked.linked_pins(), - [later].into_iter().collect(), + [later.clone()].into_iter().collect(), "reconciling main never recreates the other worktree's missing pin" ); - let degraded = linked.run(&linked.root).await; - let ExplicitOutcome::Failed { error, .. } = degraded else { - panic!("the degraded worktree fails closed in its own pass, got {degraded:?}"); + let repaired = linked.run(&linked.root).await; + let ExplicitOutcome::Completed(report) = repaired else { + panic!("the degraded worktree is repaired in its own pass, got {repaired:?}"); }; - let ReconcileError::MissingRequiredPins { missing } = error else { - panic!("expected missing required pins, got {error:?}"); - }; - assert_eq!(missing.len(), 1); - assert_eq!(missing[0].0, shared); - assert!(GitSnapshotService::new(&linked.root) - .await - .expect("snapshot") - .list_pins(&WorktreeId(linked.id.clone())) - .await - .expect("pins") - .iter() - .all(|pin| pin.tree.0 != shared)); + assert_eq!(report.repinned, 1); + assert_eq!(report.pruned_events, 0); + assert_eq!(linked.linked_pins(), [shared, later].into_iter().collect()); } diff --git a/cli/src/services/mutation_trace/runtime/ref_reconciliation.rs b/cli/src/services/mutation_trace/runtime/ref_reconciliation.rs index 50c7f63f7..facb77776 100644 --- a/cli/src/services/mutation_trace/runtime/ref_reconciliation.rs +++ b/cli/src/services/mutation_trace/runtime/ref_reconciliation.rs @@ -1,21 +1,28 @@ use std::collections::BTreeSet; -use std::path::Path; +use std::path::{Path, PathBuf}; use anyhow::Result; use super::git_snapshot::resolve_worktree_id; use crate::services::agent_trace_db::repository::RepositoryAgentTraceDb; -use crate::services::mutation_trace::store::MutationTraceStore; -use crate::services::mutation_trace::types::TreeId; +use crate::services::mutation_trace::store::{CasResult, MutationTraceStore}; +use crate::services::mutation_trace::types::{TreeId, WorktreeId}; use super::git_snapshot::{GitSnapshotService, PinInventoryError, PinnedRef}; use super::worktree_lock::{WorktreeLock, WorktreeLockError}; +pub(super) struct MaintenanceDb { + pub(super) db: RepositoryAgentTraceDb, + pub(super) path: PathBuf, +} + #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct ReconciliationReport { pub local_required: usize, pub retained: usize, pub deleted: usize, + pub repinned: usize, + pub pruned_events: usize, } #[derive(Debug, Clone, Copy, PartialEq, Eq)] @@ -27,13 +34,18 @@ pub enum ReconciliationOutcome { pub enum ReconcileError { GitDir(anyhow::Error), Lock(WorktreeLockError), - CheckoutIdentity(anyhow::Error), + WorktreeIdentity(anyhow::Error), AgentTraceDbUnavailable(anyhow::Error), SnapshotService(anyhow::Error), PinInventory(anyhow::Error), MalformedPin { ref_name: String, reason: String }, DurableRoots(anyhow::Error), MissingRequiredPins { missing: Vec }, + ObjectInventory(anyhow::Error), + UnrecoverableCursor { tree: TreeId }, + PruneTransaction(anyhow::Error), + PruneConflict, + RepinTransaction(anyhow::Error), DeleteTransaction(anyhow::Error), } @@ -55,11 +67,27 @@ impl std::fmt::Display for ReconcileError { ReconcileError::AgentTraceDbUnavailable(source) => { write!(f, "Repository Agent Trace DB is unavailable: {source}") } + ReconcileError::UnrecoverableCursor { tree } => write!( + f, + "The worktree's mutation cursor tree {} is absent from this repository's object \ + store, so its durable history does not belong to this checkout; reconciliation \ + changed nothing. Let one agent tool run in this checkout so it records a present \ + cursor, then rerun `sce doctor --fix`.", + tree.0 + ), + ReconcileError::PruneConflict => write!( + f, + "The worktree's mutation cursor advanced while unrecoverable events were being \ + pruned; reconciliation changed nothing. Rerun `sce doctor --fix`." + ), ReconcileError::GitDir(source) - | ReconcileError::CheckoutIdentity(source) + | ReconcileError::WorktreeIdentity(source) | ReconcileError::SnapshotService(source) | ReconcileError::PinInventory(source) | ReconcileError::DurableRoots(source) + | ReconcileError::ObjectInventory(source) + | ReconcileError::PruneTransaction(source) + | ReconcileError::RepinTransaction(source) | ReconcileError::DeleteTransaction(source) => write!(f, "{source}"), } } @@ -80,14 +108,14 @@ pub(super) async fn reconcile_with_held_lock( mut on_phase: H, ) -> std::result::Result where - P: std::ops::AsyncFnOnce() -> Result, + P: std::ops::AsyncFnOnce() -> Result, H: FnMut(ReconcilePhase), { let worktree_id = resolve_worktree_id(repository_root) .await - .map_err(ReconcileError::CheckoutIdentity)?; + .map_err(ReconcileError::WorktreeIdentity)?; - let mut db = open_db() + let MaintenanceDb { db, path } = open_db() .await .map_err(ReconcileError::AgentTraceDbUnavailable)?; on_phase(ReconcilePhase::DbOpened); @@ -106,7 +134,11 @@ where } })?; on_phase(ReconcilePhase::PinsInventoried); - let pinned_trees: BTreeSet = actual.iter().map(|pin| pin.tree.clone()).collect(); + let mut pinned_trees: BTreeSet = actual.iter().map(|pin| pin.tree.clone()).collect(); + + let (mut db, repair) = + repair_missing_pins(&snapshot, lock, db, &path, &worktree_id, &pinned_trees).await?; + pinned_trees.extend(repair.repinned.iter().cloned()); let store = MutationTraceStore::new(&mut db); @@ -142,5 +174,111 @@ where local_required: required_local.len(), retained: actual.len() - stale.len(), deleted: stale.len(), + repinned: repair.repinned.len(), + pruned_events: repair.pruned_events, })) } + +struct PinRepair { + repinned: Vec, + pruned_events: usize, +} + +async fn repair_missing_pins( + snapshot: &GitSnapshotService, + lock: &WorktreeLock, + mut db: RepositoryAgentTraceDb, + db_path: &Path, + worktree_id: &WorktreeId, + pinned_trees: &BTreeSet, +) -> std::result::Result<(RepositoryAgentTraceDb, PinRepair), ReconcileError> { + let required = MutationTraceStore::new(&mut db) + .load_tree_roots(worktree_id) + .await + .map_err(ReconcileError::DurableRoots)?; + let missing: Vec = required.difference(pinned_trees).cloned().collect(); + if missing.is_empty() { + return Ok(( + db, + PinRepair { + repinned: Vec::new(), + pruned_events: 0, + }, + )); + } + + let absent = snapshot + .absent_trees(&missing) + .await + .map_err(ReconcileError::ObjectInventory)?; + + let mut pruned_events = 0; + if !absent.is_empty() { + let store = MutationTraceStore::new(&mut db); + let Some(worktree_state) = store + .load_worktree_state(worktree_id) + .await + .map_err(ReconcileError::DurableRoots)? + else { + return Err(ReconcileError::MissingRequiredPins { missing }); + }; + if absent.contains(&worktree_state.cursor_tree) { + return Err(ReconcileError::UnrecoverableCursor { + tree: worktree_state.cursor_tree, + }); + } + + let revisions: Vec = store + .load_mutation_event_trees(worktree_id) + .await + .map_err(ReconcileError::DurableRoots)? + .into_iter() + .filter(|event| { + absent.contains(&event.before_tree) || absent.contains(&event.after_tree) + }) + .map(|event| event.revision) + .collect(); + + drop(db); + db = RepositoryAgentTraceDb::open_without_migrations_at(db_path) + .await + .map_err(ReconcileError::PruneTransaction)?; + match MutationTraceStore::new(&mut db) + .prune_mutation_events(worktree_id, worktree_state.revision, &revisions) + .await + .map_err(ReconcileError::PruneTransaction)? + { + CasResult::Applied => pruned_events = revisions.len(), + CasResult::Conflict => return Err(ReconcileError::PruneConflict), + } + } + + let required = MutationTraceStore::new(&mut db) + .load_tree_roots(worktree_id) + .await + .map_err(ReconcileError::DurableRoots)?; + let repinned: Vec = required.difference(pinned_trees).cloned().collect(); + let still_absent: Vec = repinned + .iter() + .filter(|tree| absent.contains(tree)) + .cloned() + .collect(); + if !still_absent.is_empty() { + return Err(ReconcileError::MissingRequiredPins { + missing: still_absent, + }); + } + + snapshot + .pin_trees(lock.lease(), worktree_id, &repinned) + .await + .map_err(ReconcileError::RepinTransaction)?; + + Ok(( + db, + PinRepair { + repinned, + pruned_events, + }, + )) +} diff --git a/cli/src/services/mutation_trace/runtime/scope_runtime.rs b/cli/src/services/mutation_trace/runtime/scope_runtime.rs index c8e220bb6..fb3397452 100644 --- a/cli/src/services/mutation_trace/runtime/scope_runtime.rs +++ b/cli/src/services/mutation_trace/runtime/scope_runtime.rs @@ -312,7 +312,7 @@ fn recovery_required( fn protected_worktree_failure(error: ProtectedWorktreeError) -> AbandonScopeError { match error { ProtectedWorktreeError::GitDirResolution(source) - | ProtectedWorktreeError::CheckoutIdentity(source) => AbandonScopeError::Other(source), + | ProtectedWorktreeError::WorktreeIdentity(source) => AbandonScopeError::Other(source), ProtectedWorktreeError::LockAcquisition(source) => { AbandonScopeError::LockAcquisition(anyhow::Error::new(source)) } diff --git a/cli/src/services/mutation_trace/store.rs b/cli/src/services/mutation_trace/store.rs index 8bdace440..68a0430ba 100644 --- a/cli/src/services/mutation_trace/store.rs +++ b/cli/src/services/mutation_trace/store.rs @@ -128,6 +128,13 @@ pub struct MutationEventPageRow { pub attribution_scope_id: Option, } +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct MutationEventTrees { + pub revision: u64, + pub before_tree: TreeId, + pub after_tree: TreeId, +} + /// The discriminant of an [`Attribution`] value. pub fn attribution_kind(attribution: &Attribution) -> AttributionKind { match attribution { @@ -256,6 +263,17 @@ const SELECT_TREE_ROOTS_BY_WORKTREE_SQL: &str = SELECT before_tree AS tree FROM mutation_trace_events WHERE worktree_id = ?1 UNION SELECT after_tree AS tree FROM mutation_trace_events WHERE worktree_id = ?1"; +const SELECT_MUTATION_EVENT_TREES_SQL: &str = + "SELECT revision, before_tree, after_tree FROM mutation_trace_events + WHERE worktree_id = ?1 + ORDER BY revision"; +const TOUCH_WORKTREE_CAS_SQL: &str = "UPDATE mutation_trace_worktrees + SET updated_at = strftime('%Y-%m-%dT%H:%M:%fZ', 'now') + WHERE worktree_id = ?1 AND revision = ?2"; +const DELETE_MUTATION_EVENT_ACTIVE_SCOPES_SQL: &str = + "DELETE FROM mutation_trace_event_active_scopes WHERE worktree_id = ?1 AND revision = ?2"; +const DELETE_MUTATION_EVENT_SQL: &str = + "DELETE FROM mutation_trace_events WHERE worktree_id = ?1 AND revision = ?2"; const SELECT_ALL_TREE_ROOTS_SQL: &str = "SELECT cursor_tree AS tree FROM mutation_trace_worktrees UNION SELECT before_tree AS tree FROM mutation_trace_events @@ -816,6 +834,67 @@ impl<'a> MutationTraceStore<'a> { Ok(rows.into_iter().collect()) } + pub async fn load_mutation_event_trees( + &self, + worktree: &WorktreeId, + ) -> Result> { + self.db + .query_map( + SELECT_MUTATION_EVENT_TREES_SQL, + (worktree.0.as_str(),), + mutation_event_trees_row_from_turso, + ) + .await + } + + pub async fn prune_mutation_events( + &mut self, + worktree: &WorktreeId, + expected_revision: u64, + revisions: &[u64], + ) -> Result { + let expected_revision_blob = encode_revision(expected_revision); + let guard = TransactionStatement::new( + TOUCH_WORKTREE_CAS_SQL, + (worktree.0.as_str(), expected_revision_blob.as_slice()), + )?; + + let revision_blobs: Vec<[u8; 8]> = revisions + .iter() + .map(|revision| encode_revision(*revision)) + .collect(); + let mut statements = Vec::with_capacity(revision_blobs.len() * 2); + for revision_blob in &revision_blobs { + statements.push(TransactionStatement::new( + DELETE_MUTATION_EVENT_ACTIVE_SCOPES_SQL, + (worktree.0.as_str(), revision_blob.as_slice()), + )?); + statements.push( + TransactionStatement::new( + DELETE_MUTATION_EVENT_SQL, + (worktree.0.as_str(), revision_blob.as_slice()), + )? + .expect_rows_affected(1), + ); + } + + let applied = self + .db + .execute_transactional_cas_batch( + "prune unrecoverable mutation-trace events", + "rerun `sce doctor --fix`", + &guard, + &statements, + ) + .await?; + + Ok(if applied { + CasResult::Applied + } else { + CasResult::Conflict + }) + } + pub async fn load_all_tree_roots(&self) -> Result> { let rows = self .db @@ -915,7 +994,10 @@ impl<'a> MutationTraceStore<'a> { Ok(rows.into_iter().next()) } - async fn load_worktree_state(&self, worktree: &WorktreeId) -> Result> { + pub async fn load_worktree_state( + &self, + worktree: &WorktreeId, + ) -> Result> { let rows = self .db .query_map( @@ -1212,6 +1294,23 @@ fn scope_provenance_row_from_turso(row: &turso::Row) -> Result }) } +fn mutation_event_trees_row_from_turso(row: &turso::Row) -> Result { + let revision_blob: Vec = row + .get(0) + .context("failed to read mutation_trace_events.revision")?; + let before_tree: String = row + .get(1) + .context("failed to read mutation_trace_events.before_tree")?; + let after_tree: String = row + .get(2) + .context("failed to read mutation_trace_events.after_tree")?; + Ok(MutationEventTrees { + revision: decode_revision(&revision_blob)?, + before_tree: TreeId(before_tree), + after_tree: TreeId(after_tree), + }) +} + fn mutation_event_page_row_from_turso(row: &turso::Row) -> Result { let revision_blob: Vec = row .get(0) diff --git a/context/architecture.md b/context/architecture.md index 14b3f9a1f..3d48c6a72 100644 --- a/context/architecture.md +++ b/context/architecture.md @@ -135,7 +135,7 @@ The repository includes a new placeholder Rust binary crate at `cli/`. - `cli/src/services/config/types.rs` is the canonical owner for the shared runtime/config primitive seam used by the CLI: `LogLevel`, `LogFormat`, `SCE_LOG_LEVEL`, `SCE_LOG_FORMAT`, `SCE_LOG_DIR`, `DEFAULT_LOG_FILE_RETENTION_LIMIT`, and the shared bool parsing helpers used by both config resolution and observability bootstrap; `cli/src/services/config/mod.rs` re-exports those primitives through the facade. - `cli/src/services/capabilities.rs` defines the current broad CLI capability traits consumed by the borrowed, compile-time-typed `AppContext`: `FsOps` with `StdFsOps` as a method-less filesystem capability marker and `GitOps` with `ProcessGitOps` for git command execution and availability checks. Existing service internals do not consume these traits directly yet; command execution uses narrow accessors and repo-root-scoped context derivation. - `cli/src/services/lifecycle.rs` defines the current compile-safe lifecycle seam. `ServiceLifecycle` has default no-op generic `diagnose`, `fix`, and `setup` methods over `C: HasRepoRoot`, with lifecycle-owned health, fix, and setup result types so the trait contract is not publicly anchored to doctor/setup module types or the full `AppContext` shape. The same module owns the static `LifecycleProvider` enum and shared `lifecycle_providers(include_hooks)` catalog/factory, returning providers in deterministic order (config → local_db → auth_db → agent_trace_db → hooks when requested); enum dispatch calls each concrete provider through generic context methods without boxed lifecycle-provider allocation or repo-root trait-object context erasure. Hooks exposes a `HooksLifecycle` provider in `cli/src/services/hooks/lifecycle.rs` for hook rollout diagnosis/fix/setup using lifecycle-owned health records plus the canonical required-hook installer. Config exposes a `ConfigLifecycle` provider in `cli/src/services/config/lifecycle.rs` for global/repo-local config validation and repo-local `.sce/config.json` bootstrap. local_db exposes a `LocalDbLifecycle` provider in `cli/src/services/local_db/lifecycle.rs` for canonical local DB path health, parent-directory readiness/bootstrap, and `LocalDb::new()` setup. auth_db exposes an `AuthDbLifecycle` provider in `cli/src/services/auth_db/lifecycle.rs` for canonical auth DB path health, parent-directory readiness/bootstrap, and `AuthDb::new()` setup. agent_trace_db exposes an `AgentTraceDbLifecycle` provider in `cli/src/services/agent_trace_db/lifecycle.rs` for setup-time repository-scoped Agent Trace storage initialization when a repo root is available and repository Agent Trace DB path health/fix from resolved repository identity, returning an actionable "requires a Git repository" diagnostic outside repository context (no global/checkout fallback path; the former fallback was removed by the `retire-legacy-agent-trace-db` plan). Doctor runtime aggregates the full provider catalog for `diagnose` and `fix` and adapts lifecycle records into doctor report/fix records at the orchestration boundary; setup command aggregates the shared catalog for `setup` with hooks included only when requested and adapts hook setup outcomes before rendering setup-owned messages. -- Agent Trace lifecycle setup resolves `agent_trace.repository_id` / `agent_trace.repository_remote`, creates/reuses checkout identity for diagnostics, and creates or migrates the repository-scoped DB through `agent_trace_storage::resolve_agent_trace_storage(...)`; hook runtime uses the same storage identity/path resolution and no-migration open path, with missing or stale schema failing open through the existing `Run 'sce setup'.` guidance. +- Agent Trace lifecycle setup resolves `agent_trace.repository_id` / `agent_trace.repository_remote` and creates or migrates the repository-scoped DB through `agent_trace_storage::resolve_agent_trace_storage(...)`; hook runtime uses the same storage identity/path resolution and no-migration open path, with missing or stale schema failing open through the existing `Run 'sce setup'.` guidance. - `cli/src/services/auth_command/mod.rs` defines the implemented auth command surface for `sce auth login|logout|whoami`, including device-flow login, stored-credential validation/renewal through login with device-flow fallback, logout, and Control Plane `/me`-backed whoami rendering in text/JSON formats; text mode uses flat `Email`, `First Name`, `Last Name`, `Role`, `Permissions`, and `Organization Name` labels, with optional names and missing role/permissions/workspace values handled deterministically. Logged-out text returns exact login guidance and renewal reports retain the `login` operation label. `cli/src/services/auth_command/command.rs` owns the `AuthCommand` payload used by the static `RuntimeCommand` enum. There is no public renewal or status subcommand. - `cli/src/services/db/mod.rs` provides shared async Turso infrastructure: `DbSpec` supplies a service name, path, ordered migrations, and config key; `TursoDb` / `EncryptedTursoDb` own concrete connections but no Tokio runtime. Construction, migration, query/materialization, transaction, readiness, and checkpoint operations are awaited through `run_with_retry` on the application runtime. SQL, one-connection transaction scope, migration metadata, retry classification, WAL, encryption, and per-database policy remain adapter-owned; see `context/sce/shared-turso-db.md`. - `cli/src/services/local_db/mod.rs` provides `LocalDb = TursoDb` with zero local migrations; inherited construction, `execute`, `query`, and `query_map` operations are async and use the shared adapter retry policies. @@ -155,7 +155,7 @@ The repository includes a new placeholder Rust binary crate at `cli/`. - `sce sync [--format text|json]` is implemented: `cli/src/services/sync/sync.rs` resolves repository-scoped Agent Trace storage, authenticates against the control plane with stored WorkOS credentials, uses the config-resolved `control_plane_base_url` with baked default `https://sce.crocoderlab.dev`, calls the ingestion `/state` endpoint once, then starts the three remote `messages`/`parts`/`agent_traces` stream state machines concurrently (`diff_traces` remains a local-only compatibility report) via `AgentTraceExportReader` and a shared per-stream reconciliation engine. Batches and cursor refreshes remain sequential within each stream, while fixed stream order is retained for final and stream-completion reporting; `cli/src/services/sync/render_sync.rs` renders the converged `AgentTraceSyncReport` as concise per-stream text or `camelCase` JSON without a nested subcommand field (see `context/cli/sync-command.md`). Local DB bootstrap and setup-time repository-scoped Agent Trace DB initialization otherwise still flow through lifecycle providers aggregated by setup, while repository-scoped DB health/repair flows through the doctor surface. The former trace database inspection and nested sync surfaces are unavailable. - `cli/src/services/patch.rs` defines the standalone patch domain model (`ParsedPatch`, `PatchFileChange`, `FileChangeKind`, `PatchHunk`, `TouchedLine`, `TouchedLineKind`) for in-memory parsed unified-diff representation, capturing only touched lines (added/removed) plus minimal per-file/per-hunk metadata while excluding non-hunk headers and unchanged context lines. All types are `serde`-serializable/deserializable with `snake_case` JSON field naming. The module also provides `parse_patch`, a public parser function that converts raw unified-diff text (both `Index:` SVN-style and `diff --git` git-style formats) into `ParsedPatch` structs, with `ParseError` for actionable malformed-input diagnostics. Storage-agnostic JSON load helper (`load_patch_from_json` for string input) reconstruct `ParsedPatch` from serialized JSON content with `PatchLoadError` for actionable deserialization diagnostics. Its patch-set operations now include deterministic ordered combination plus target-shaped intersection that prefers exact touched-line matches and falls back to historical `kind`+`content` matching when incremental diffs and canonical post-commit diffs have drifted line numbers; `parse_patch`, `combine_patches`, and `intersect_patches` are consumed by the active post-commit hook runtime. - `cli/src/services/structured_patch.rs` defines the synchronous structured editor-hook derivation seam. It derives Claude `PostToolUse` `Write` structured-update hunks, `Write` `tool_input.content` create fallback, and `Edit` structured-patch payloads into canonical `ParsedPatch` values plus Claude session/tool metadata, returning deterministic skip reasons for unsupported events/tools/payload shapes. The module is pure and side-effect-free. It is wired into `sce hooks diff-trace` for Claude payload classification at intake (T04) and into `RepositoryAgentTraceDb::recent_diff_trace_patches` for post-commit structured payload parsing dispatch at read time (T05). -- `cli/src/services/` contains module boundaries for command_registry, lifecycle, auth_command, config, setup, doctor, hooks, checkout identity, bash_policy, version, completion, help, patch, sync, SCE web URL helpers, shared database infrastructure, local DB adapters, encrypted auth DB adapters, and Agent Trace DB adapters with explicit trait seams for future implementations. `cli/src/services/checkout/` owns checkout ID file infrastructure (the per-checkout Agent Trace DB opener/path helper was removed by the `retire-legacy-agent-trace-db` plan); active setup/hooks use `agent_trace_storage` to establish checkout identity as diagnostics and initialize/open the repository-scoped DB, while `sce doctor` surfaces checkout identity facts plus credential-safe repository Agent Trace DB metadata. There is no checkout-scoped discovery or former trace inspection surface; any pre-migration `agent-trace-*.db` files on disk are never touched and no longer inspectable via the CLI. `cli/src/services/bash_policy.rs` owns both the CLI-agnostic evaluator logic and the hidden `sce policy bash` command adapter used by OpenCode, Claude, and Pi callers; Codex delegates in-process to the same evaluator for its native `PreToolUse(Bash)` response. `cli/src/services/command_registry.rs` defines the static `RuntimeCommand` enum, deterministic `CommandRegistry` name catalog, and `build_default_registry()` function for command dispatch metadata. Service-owned command modules own the runtime command payload structs for help/help-text, version, completion, auth, config, setup, doctor, hooks, policy, and sync. +- `cli/src/services/` contains module boundaries for command_registry, lifecycle, auth_command, config, setup, doctor, hooks, bash_policy, version, completion, help, patch, sync, SCE web URL helpers, shared database infrastructure, local DB adapters, encrypted auth DB adapters, and Agent Trace DB adapters with explicit trait seams for future implementations. The former `cli/src/services/checkout/` checkout-identity module was removed by the `remove-checkout-id` plan (see `context/cli/checkout-identity.md`); there is no per-clone identity, worktrees are identified by Git topology (`resolve_worktree_id`), active setup/hooks use `agent_trace_storage` to initialize/open the repository-scoped DB, and `sce doctor` surfaces credential-safe repository Agent Trace DB metadata. There is no checkout-scoped discovery or former trace inspection surface; any pre-migration `agent-trace-*.db` files on disk are never touched and no longer inspectable via the CLI. `cli/src/services/bash_policy.rs` owns both the CLI-agnostic evaluator logic and the hidden `sce policy bash` command adapter used by OpenCode, Claude, and Pi callers; Codex delegates in-process to the same evaluator for its native `PreToolUse(Bash)` response. `cli/src/services/command_registry.rs` defines the static `RuntimeCommand` enum, deterministic `CommandRegistry` name catalog, and `build_default_registry()` function for command dispatch metadata. Service-owned command modules own the runtime command payload structs for help/help-text, version, completion, auth, config, setup, doctor, hooks, policy, and sync. - `cli/README.md` is the crate-local onboarding and usage source of truth for placeholder behavior, safety limitations, and roadmap mapping back to service contracts. - `flake.nix` applies `rust-overlay` (`oxalica/rust-overlay`) to nixpkgs, pins `rust-bin.stable.1.95.0.default` with `rustfmt` + `clippy`, and reads package/check version from repo-root `.version`. Its `cliGeneratedInput` derivation invokes the shared generated-input producer from a declarative source containing the producer and canonical inputs, then publishes the producer-validated handoff as its store output. Native, release, test, and Clippy Cargo derivations receive that same store path through `SCE_CLI_GENERATED_INPUT_DIR`; their Cargo environments exclude Pkl and assert that it is unavailable. Repository-mode `cli/build.rs` validates the handoff before copying it into `OUT_DIR`; published crates use the validated packaging-only fallback. The build script stages SQL under `OUT_DIR/static/migrations` and writes `OUT_DIR/generated_migrations.rs` with deterministic migration constants sorted by numeric filename prefix. - Crane dependency-only derivations and `cli-fmt` intentionally do not receive `SCE_CLI_GENERATED_INPUT_DIR`, so canonical generation changes invalidate the producer and compiling derivations while preserving host/musl dependency artifacts and formatting. The root flake runs `cli-tests`, `cli-clippy`, and `cli-fmt` through Crane-backed paths and exposes `cli-generated-input` as the focused payload/inventory integrity check. It also exposes directory-scoped JS validation derivations for `npm/` and `config/lib/`, while `pkl-generated` uses a narrow canonical-input source set plus maybe-missing forbidden paths so reintroduced generated repository artifacts fail the check. diff --git a/context/cli/doctor-ref-reconciliation.md b/context/cli/doctor-ref-reconciliation.md index 1d53a602c..26dacdb14 100644 --- a/context/cli/doctor-ref-reconciliation.md +++ b/context/cli/doctor-ref-reconciliation.md @@ -29,11 +29,15 @@ validity, and appends one fix-result row (category `mutation_scope_health`): | Pass result | Outcome | Detail | | --- | --- | --- | -| `Completed` | `fixed` | deleted / retained / locally required counts | +| `Completed` | `fixed` | deleted / retained / locally required counts, plus pruned-event and re-pinned counts when non-zero (JSON `counts` always carries `repinned` and `pruned_events`) | | `CompletedStatePersistFailed` | `fixed` | counts plus a diagnostics-not-recorded warning | -| `Failed` | `failed` | typed kind (for example `agent_trace_db_missing`, `agent_trace_db_incompatible_schema`, `agent_trace_db_missing_metadata`, `agent_trace_db_repository_mismatch`, `malformed_ref`, `missing_required_pins`, `durable_roots_unavailable`, `delete_transaction_failed`) and message, plus any state warning | +| `Failed` | `failed` | typed kind (for example `agent_trace_db_missing`, `agent_trace_db_incompatible_schema`, `agent_trace_db_missing_metadata`, `agent_trace_db_repository_mismatch`, `malformed_ref`, `missing_required_pins`, `object_inventory_failed`, `unrecoverable_cursor`, `prune_transaction_failed`, `prune_conflict`, `repin_transaction_failed`, `durable_roots_unavailable`, `delete_transaction_failed`) and message, plus any state warning | | `Skipped(Busy)` | `skipped` | another boundary or pass holds the worktree lock; retry | +An `unrecoverable_cursor` failure changes nothing, and its message tells the +operator to let one agent tool run in this checkout, which commits a present +cursor, and then rerun `sce doctor --fix`. + A busy pass leaves state unchanged and is not a failure. While a pass runs, a mutation boundary may wait up to the 10 s coordinator lock timeout. diff --git a/context/cli/mutation-trace-missing-history-recovery.md b/context/cli/mutation-trace-missing-history-recovery.md new file mode 100644 index 000000000..8d8b491a1 --- /dev/null +++ b/context/cli/mutation-trace-missing-history-recovery.md @@ -0,0 +1,80 @@ +# Missing-pin repair and missing durable history (`ref_reconciliation::repair_missing_pins`) + +The repair step for the target worktree `W` in [snapshot-ref reconciliation](mutation-trace-ref-reconciliation.md), +run by `sce doctor --fix` before the local-consistency check. + +## Repair steps + +The repository-scoped DB is keyed by repository identity, not by clone, and the +primary worktree is always `main`. A re-clone of the same remote therefore +inherits the old clone's `main` cursor and events while its pins and snapshot +objects stayed in the deleted `.git`. Before the local-consistency check, +`repair_missing_pins` handles `W`'s unpinned durable roots: + +1. Classify each unpinned root with `git cat-file --batch-check` + (`GitSnapshotService::absent_trees`). A non-tree object fails the pass with + `ObjectInventory`. +2. If any root is absent from the object store and it is `W`'s own + `cursor_tree`, fail with `UnrecoverableCursor` and change nothing. The message + tells the operator to let one agent tool run in this checkout and rerun + `sce doctor --fix` (see [case 2](#missing-durable-history-heuristic)). +3. Otherwise delete every `W` event whose `before_tree` or `after_tree` is absent, + with its `mutation_trace_event_active_scopes` rows, in one transaction guarded + by a CAS on `W`'s current revision (`MutationTraceStore::prune_mutation_events`). + A lost CAS is `PruneConflict`. Such events can never be replayed by + attribution, which already treats unreadable trees as a reconstruction + barrier. The verified maintenance connection is read-only and Turso shares + one database handle per path in a process, so the read-only connection is + dropped and a writable one is opened on the same verified path only when a + prune is needed. +4. Re-pin every remaining unpinned root (all present) in one atomic + `git update-ref --no-deref --stdin` `create` transaction + (`GitSnapshotService::pin_trees`). + +Order makes the repair idempotent: a crash after the prune and before the re-pin +leaves only present unpinned roots, which the next pass re-pins. + +## Missing durable history (heuristic) + +Detection is a heuristic over the object store, built by the +`missing-history-recovery` plan. A durable tree absent from the current Git +object store is history this checkout cannot have produced or kept. It detects +**missing history, not clone identity**: no checkout ID, clone ID, marker, or +identity column exists, and `resolve_worktree_id` still returns `main` for the +primary worktree of every clone. +An existing tree is **not evidence of an unchanged clone**; a tree can exist +because it was pushed, fetched, or recreated from identical content. Concurrent +independent clones sharing one DB are out of scope. + +Three cases for the target worktree `W`: + +1. **Missing non-cursor trees.** Every `W` event whose `before_tree` or + `after_tree` is absent is pruned with its active-scope rows, and the surviving + unpinned roots are re-pinned. The pass completes and reports `pruned_events` + and `repinned`. +2. **Missing cursor tree.** The pass fails closed with `unrecoverable_cursor` and + changes nothing. The state is transient: the next committed agent hook + boundary replaces `cursor_tree` with a freshly captured, pinned tree and the + old cursor becomes an event `before_tree`, which case 1 then prunes on the + next `sce doctor --fix`. No separate cursor-recovery transition exists. +3. **Existing cursor and event trees.** No destructive action and no claim that + the clone is unchanged; reconciliation continues under the limitation above. + +Repair runs only from the explicit `sce doctor --fix` pass. No agent hook +boundary prunes events, rewrites the cursor, or abandons scopes because of +absent trees, and plain `sce doctor` does not detect absent trees; it keeps the +existing reconciliation recommendation. + +Invariants: + +- **I1:** After a completed pass, every durable root of `W` (cursor and event + trees) is present in the object store and pinned under `W`'s namespace. +- **I2:** An event is pruned only if one of its trees is absent; every event whose + trees are all present survives. +- **I3:** The pass never changes `W`'s cursor; an absent cursor fails closed with + nothing changed. +- **I4:** A failure at any step leaves either the pre-pass state or a state the + next pass repairs (a prune committed before a failed re-pin leaves only present + unpinned roots). +- **I5:** Stale-pin deletion keeps its repository-wide rule: a pin is deleted only + if its tree is a durable root of no worktree. diff --git a/context/cli/mutation-trace-protected-worktree.md b/context/cli/mutation-trace-protected-worktree.md index d8a61bf9a..9bd2abac8 100644 --- a/context/cli/mutation-trace-protected-worktree.md +++ b/context/cli/mutation-trace-protected-worktree.md @@ -95,12 +95,12 @@ map it onto its own error surface without losing which safety step failed: | `LockAcquisition(WorktreeLockError)` | lock acquire/timeout | untouched | | `ExternalTaintMarker { operation: Inspect \| Persist, source }` | fence inspect/arm | left as it was | | `MarkerWorkerFailed(JoinError)` | arming blocking worker panicked or was cancelled by the runtime while holding the lock | unknown; never auto-cleared | -| `CheckoutIdentity(anyhow::Error)` | Git-derived worktree identity resolution, after the fence is armed | **armed** | +| `WorktreeIdentity(anyhow::Error)` | Git-derived worktree identity resolution, after the fence is armed | **armed** | `ExternalTaintOperation` lives here, beside the fence step that produces it, and is re-exported by `coordinator.rs` so `CoordinateError::ExternalTaintMarker` keeps naming it; `AbandonScopeError::ExternalTaintMarker` carries the same type. -`coordinate()` maps `GitDirResolution` and `CheckoutIdentity` +`coordinate()` maps `GitDirResolution` and `WorktreeIdentity` onto `CoordinateError::Other`, `LockAcquisition` onto `CoordinateError::LockAcquisition`, and the fence variant onto `CoordinateError::ExternalTaintMarker` with the same `operation` — the exact diff --git a/context/cli/mutation-trace-ref-reconciliation.md b/context/cli/mutation-trace-ref-reconciliation.md index 6c53a0809..58e77f184 100644 --- a/context/cli/mutation-trace-ref-reconciliation.md +++ b/context/cli/mutation-trace-ref-reconciliation.md @@ -90,7 +90,7 @@ Identity is Git-topology-derived (`resolve_worktree_id`): `main` for the primary worktree, `worktrees/` for a linked worktree. Failure is an `Err`. The former checkout-identity service was removed (see [`checkout-identity.md`](checkout-identity.md)); there is no skip-on-missing-identity -outcome. `ReconcileError::CheckoutIdentity` remains the error variant for a +outcome. `ReconcileError::WorktreeIdentity` is the error variant for a worktree identity that cannot be resolved. The dedicated DB opener and the `ref-maintenance.json` state file are documented in @@ -109,8 +109,9 @@ flowchart TD inv["list_pins(W) — actual pins under refs/sce/mutation-cursor/<W>/"] local["load_tree_roots(W)\n(this worktree's cursor + event trees)"] repo["load_all_tree_roots()\n(every worktree's cursor + event trees)"] - inv --> lc - local --> lc{"local consistency:\ndurable_roots(W) ⊆ pinned_trees(W)?"} + inv --> rep["missing-pin repair (W only):\nprune events with absent trees,\nre-pin present trees"] + local --> rep + rep --> lc{"local consistency:\ndurable_roots(W) ⊆ pinned_trees(W)?"} lc -- "no" --> fail["ReconcileError::MissingRequiredPins\n— fail closed, delete nothing"] lc -- "yes" --> ds inv --> ds{"deletion safety:\npin.tree ∉ durable_roots(repository)?"} @@ -120,16 +121,25 @@ flowchart TD ``` - **Local consistency** (`load_tree_roots(W)`) is per-worktree and checked before - any deletion. If it fails the pass deletes nothing and returns - `MissingRequiredPins`. A missing pin in some *other* worktree never makes `W`'s - pass fail. + any deletion, after the [missing-pin repair](mutation-trace-missing-history-recovery.md). If it still + fails the pass deletes nothing and returns `MissingRequiredPins`. A missing pin + in some *other* worktree never makes `W`'s pass fail. - **Deletion safety** (`load_all_tree_roots()`) is repository-wide: `delete(W, T)` only if `T` is in **no** worktree's durable root set. A pin another worktree requires is retained. Claimed: these two invariants. **Not claimed:** that every repository-wide durable root is pinned. Reconciliation does not repair, recreate, or verify missing pins -belonging to other worktrees. +belonging to other worktrees; each worktree is repaired only by its own pass. + +## Missing-pin repair + +Before the local-consistency check, `repair_missing_pins` prunes `W` events whose +trees are absent from the object store and re-pins the surviving unpinned roots; +an absent `W` cursor fails closed with `UnrecoverableCursor`. Detection is a +heuristic: it finds missing history, not clone identity, and an existing tree is +not evidence of an unchanged clone. Steps, the three cases, and invariants I1–I5 +live in [`mutation-trace-missing-history-recovery.md`](mutation-trace-missing-history-recovery.md). A DB `TreeId` is a *logical* durability requirement, not itself a Git reachability edge: it obliges reconciliation to keep at least one SCE ref @@ -156,18 +166,23 @@ Every step runs under the lock and every fallible step maps to one dedicated | --- | --- | | `resolve_git_dir` | `GitDir` | | lock acquisition (`EXPLICIT_LOCK_TIMEOUT`) | `Lock(WorktreeLockError)`; surfaced as `Skipped(Busy)` by `reconcile_explicit` | -| worktree identity resolution | `CheckoutIdentity` | +| worktree identity resolution | `WorktreeIdentity` | | verified-existing DB open | `AgentTraceDbUnavailable` (downcastable `ExistingRepositoryDbError`) | | `GitSnapshotService::new` | `SnapshotService` | | `list_pins` → `PinInventoryError::Git` | `PinInventory` | | `list_pins` → `PinInventoryError::MalformedRef` | `MalformedPin { ref_name, reason }`, delete nothing | | `load_tree_roots` / `load_all_tree_roots` | `DurableRoots` (durable-root queries belong to migration `004_mutation_trace_protocol`) | -| a target-worktree root has no pin | `MissingRequiredPins { missing }`, delete nothing | +| `cat-file --batch-check` of unpinned roots | `ObjectInventory`, change nothing | +| `W`'s `cursor_tree` is absent from the object store | `UnrecoverableCursor { tree }`, change nothing | +| writable open or prune transaction | `PruneTransaction`, prune nothing (atomic-or-nothing) | +| prune CAS lost to a concurrent revision | `PruneConflict`, prune nothing | +| re-pin transaction | `RepinTransaction`, pin nothing; events already pruned stay pruned | +| a target-worktree root still has no pin after repair | `MissingRequiredPins { missing }`, delete nothing | | `delete_pins` transaction | `DeleteTransaction`, delete nothing (atomic-or-nothing) | `AgentTraceDbUnavailable` here is a **maintenance** error only: reconciliation -never arms `ExternalTaintMarker`, calls `protocol::*`, or writes a -`mutation_trace_*` row, and never becomes +never arms `ExternalTaintMarker` or calls `protocol::*`, writes `mutation_trace_*` +rows only through the CAS-guarded prune of unrecoverable events, and never becomes `CoordinateError::AgentTraceDbUnavailable` (contrast [`mutation-trace-external-taint.md`](mutation-trace-external-taint.md)). `delete_pins` is the only ref-deletion path; there is no `git gc`, object @@ -178,9 +193,10 @@ removal, or independent deletion. `reconcile_with_held_lock` returns `Result` where `ReconciliationOutcome` has the single variant `Reconciled(ReconciliationReport)`. -`ReconciliationReport { local_required, retained, deleted }`: -`local_required = load_tree_roots(W).len()`, `deleted` = stale pins removed, -`retained = actual.len() − deleted`. `retained == local_required` is not an +`ReconciliationReport { local_required, retained, deleted, repinned, pruned_events }`: +`local_required = load_tree_roots(W).len()` after repair, `deleted` = stale pins +removed, `retained = actual.len() − deleted` (pre-repair inventory), +`repinned` = pins recreated by the repair, `pruned_events` = events pruned. `retained == local_required` is not an invariant; only `report.local_required ≤ report.retained` holds. `reconcile_explicit` returns the typed, never-merged `ExplicitOutcome`: diff --git a/context/cli/mutation-trace-runtime-coordinator.md b/context/cli/mutation-trace-runtime-coordinator.md index b0845aee5..b05b4c988 100644 --- a/context/cli/mutation-trace-runtime-coordinator.md +++ b/context/cli/mutation-trace-runtime-coordinator.md @@ -112,8 +112,11 @@ see Testing boundary). Only harness/command wiring remains. `reconcile_explicit` (single `WorktreeLock` acquisition, verified-existing Agent Trace DB, state recorded under the same lock) is its only production entrypoint, called by `sce doctor --fix`. It deletes only pins whose tree is a - durable root of **no** worktree, fails closed if any local root lacks a pin, - and writes no `mutation_trace_*` row or taint marker. Only the namespace of the + durable root of **no** worktree, first repairs local roots that lack a pin + (prunes events whose snapshot trees are absent from the object store, re-pins + present trees, fails closed if the worktree's own cursor tree is absent), and + writes no taint marker. Its only `mutation_trace_*` writes are that + CAS-guarded prune. Only the namespace of the currently resolvable worktree is inventoried; refs owned by removed linked worktrees survive (deferred follow-up). Full contract in [`mutation-trace-ref-reconciliation.md`](mutation-trace-ref-reconciliation.md). diff --git a/context/context-map.md b/context/context-map.md index e1e9f3e67..2a258a989 100644 --- a/context/context-map.md +++ b/context/context-map.md @@ -34,14 +34,15 @@ Feature/domain context: - `context/cli/mutation-trace-store.md` (durable persistence for the mutation-cursor protocol in `cli/src/services/mutation_trace/store.rs`, built by the `mutation-cursor-store-persistence` plan: the one-directional `protocol.rs` -> `DurableTransition::between` (pure structural diff) -> `store.rs` (SQL translation) -> `RepositoryAgentTraceDb` boundary; migration `004_mutation_trace_protocol.sql`'s five tables; migration `006_mutation_trace_health_invariant.sql`'s transactional rebuild of the worktree and event tables with a cross-column `tainted`/`failure_kind` health `CHECK`, `failure_kind`-authoritative legacy normalization, and a re-runnable SQL body, plus the shared fail-closed `validate_health_encoding` used by all three health-row decoders; `AttemptState`/`external_taint` non-persistence; the 8-byte big-endian `BLOB` revision encoding plus explicit non-`Debug` enum codecs; the bounded hot-path `load_worktree` read vs. the cold-path `load_mutation_event` and descending, exact-worktree, cursor-paged `load_mutation_event_page` reader capped at 32 rows; the public cold-path single-row `load_scope` scope seam that returns one `mutation_trace_scopes` row as `Option` without widening into a projection and, unlike `load_worktree`, never adjudicates worktree identity (a cross-worktree scope is returned as-is, and the mismatch is the caller's decision); the cold-path read-only `load_tree_roots` (one worktree) / `load_all_tree_roots` (repository-wide) durable-tree-SHA queries for ref reconciliation, each a single-statement `UNION` of `cursor_tree`/`before_tree`/`after_tree` read from one DB snapshot; `commit`'s single-`BEGIN IMMEDIATE` CAS batch via `TursoDb::execute_transactional_cas_batch`, distinguishing `Conflict`/retryable-transient/deterministic-`Err` outcomes; and the store's non-goals — no Git/filesystem I/O, no attribution/boundary-kind decisions, no retry-after-`Conflict` loop, no row deletion) - `context/cli/mutation-scope-provenance.md` (the insert-once `mutation_trace_scope_provenance` table from migration `005_mutation_scope_provenance.sql` and its `MutationTraceStore::register_scope_provenance` / `load_scope_provenance` seams: `scope_id -> session_id + model_id?` metadata *about* an established scope, stored beside but never inside protocol state — outside `ProtocolState` and the CAS batch, and never an input to `IneligibleUnscoped` / `AiExclusive` / `AiContended`; `session_id` is immutable identity and `model_id` immutable first-observed metadata, so the first persisted row wins in both directions and only a differing `session_id` for an existing `scope_id` errors; the supported `MutationTraceStore` write seam requires an existing owning `mutation_trace_scopes` row and never creates one implicitly, enforced in the store rather than by a `FOREIGN KEY`, leaving `004_mutation_trace_protocol.sql` unchanged; creation is additionally admission-bounded in the runtime — a row may be created only while the durable scope is `NeverSeen`, so after admission absent provenance stays absent permanently and a later `Start` replay cannot backfill it, while an existing row is still validated on every provenance-carrying `Start`; producers own canonicalization — both Codex and Claude adapters supply provenance today, with Claude resolving exact `(cc_, agent_id)` model state at admission and degrading unavailable models to `NULL`) - `context/cli/mutation-trace-runtime-coordinator.md` (imperative-shell runtime layer in `cli/src/services/mutation_trace/runtime/`, built by the `mutation-cursor-runtime-coordinator` plan: the per-worktree OS advisory lock, `runtime::worktree_lock::WorktreeLock::acquire_async(git_dir, timeout)` at `/sce/mutation-cursor.lock`, bounded `try_lock()` polling with a distinct matchable timeout error and RAII release, and its distinction from the separate checkout-identity-creation lock; the isolated Git snapshot service `runtime::git_snapshot::GitSnapshotService` (documented in `mutation-trace-snapshot-service.md`); and `runtime::coordinator`'s protocol-integration pipeline (`RuntimeBoundary`/`CoordinateOutcome`/`CoordinateError`, the load → recover-if-needed → prepare/commit CAS-retry loop, and its own bounded snapshot-failure taint-retry loop) plus the public `coordinate(repository_root, boundary, open_db)` entrypoint that runs the shared `runtime::protected_worktree` prefix (`WorktreeLock` -> external-taint write-ahead fence -> `WorktreeId`, extracted so a second entrypoint cannot drift from it and documented in `mutation-trace-protected-worktree.md`), invokes the caller-supplied `open_db` provider so DB acquisition falls inside that fence, drives the pipeline under one held lock, and clears the marker through `ProtectedWorktree::complete()` only on success; an inherited external-taint marker is overlaid onto `protocol::database_failure` recovery on the next invocation, against the single captured snapshot and re-injected across a losing recovery CAS; the private `runtime::ref_reconciliation` per-worktree snapshot-ref maintenance pass (documented in `mutation-trace-ref-reconciliation.md`) shares the same `WorktreeLock`; `coordinate()` and `abandon_scope()` are now `pub(crate)` re-exported from `runtime/mod.rs` (documented in `mutation-scope-runtime.md`) while the runtime submodules themselves stay private; a generic `sce hooks mutation-scope` CLI ingress now drives both entrypoints, joined by current Claude Code and Codex adapter drivers (registered by `sce setup`/`sce setup --codex` and reachable) and an OpenCode adapter driver (lifecycle + recovery wired, and now generated as the final `sce-mutation-scope.ts` plugin and installed by `sce setup`), documented in `mutation-scope-runtime.md`) -- `context/cli/mutation-trace-protected-worktree.md` (the shared safety prefix every mutation-cursor runtime entrypoint runs behind, in `cli/src/services/mutation_trace/runtime/protected_worktree.rs`, extracted from `coordinate()` by the `mutation-scope-runtime-integration` plan so a second entrypoint cannot drift from it: `ProtectedWorktree::acquire(repository_root)` running the safety-critical fixed order resolve `git_dir` → `WorktreeLock` (module-owned 10s `WORKTREE_LOCK_TIMEOUT`) → `ExternalTaintMarker::exists()` → `persist()` (fence armed write-ahead of every fallible step that follows, including DB acquisition) → `get_or_create_checkout_id` as `WorktreeId`; the `worktree_id()` / `inherited_external_taint()` / consuming `complete()` surface, where `complete()` clears the marker under the still-held lock and is the only thing that ever clears it while `Drop` releases only the lock; and the one-variant-per-prefix-step `ProtectedWorktreeError` (`GitDirResolution` | `LockAcquisition` | `ExternalTaintMarker { operation, source }` | `CheckoutIdentity`) each entrypoint maps onto its own error surface — `coordinate()` onto exactly the `CoordinateError` variants that step produced before the extraction) +- `context/cli/mutation-trace-protected-worktree.md` (the shared safety prefix every mutation-cursor runtime entrypoint runs behind, in `cli/src/services/mutation_trace/runtime/protected_worktree.rs`, extracted from `coordinate()` by the `mutation-scope-runtime-integration` plan so a second entrypoint cannot drift from it: `ProtectedWorktree::acquire(repository_root)` running the safety-critical fixed order resolve `git_dir` → `WorktreeLock` (module-owned 10s `WORKTREE_LOCK_TIMEOUT`) → `ExternalTaintMarker::exists()` → `persist()` (fence armed write-ahead of every fallible step that follows, including DB acquisition) → `resolve_worktree_id` as `WorktreeId`; the `worktree_id()` / `inherited_external_taint()` / consuming `complete()` surface, where `complete()` clears the marker under the still-held lock and is the only thing that ever clears it while `Drop` releases only the lock; and the one-variant-per-prefix-step `ProtectedWorktreeError` (`GitDirResolution` | `LockAcquisition` | `ExternalTaintMarker { operation, source }` | `WorktreeIdentity`) each entrypoint maps onto its own error surface — `coordinate()` onto exactly the `CoordinateError` variants that step produced before the extraction) - `context/cli/mutation-trace-scope-abandonment.md` (the mutation-cursor runtime's second protected entrypoint in `cli/src/services/mutation_trace/runtime/scope_runtime.rs`, built by the `mutation-scope-runtime-integration` plan and the first production call site for `protocol::abandon`: `abandon_scope(repository_root, scope, open_db) -> Result` retires a scope whose final worktree boundary was never observed, sharing `coordinate()`'s `ProtectedWorktree` prefix but deliberately capturing **no** Git snapshot, pin, diff, reconciliation, scope registration, or worktree initialization — so abandonment is not a `RuntimeBoundary` and needs no Quint change; the classify-before-transition order that recovers what `protocol::abandon`'s uniform guarded no-op cannot report (`load_scope` first for the missing-row and cross-worktree cases the projection seam treats as errors, then `load_worktree` for the `NeverSeen` / terminal / `Active` split); the `Abandoned` / `AlreadyTerminal` / `RecoveryRequired` outcomes and the `InheritedExternalTaint` | `MissingScope` | `NeverSeenScope` | `MissingWorktreeState` recovery reasons; the inherited-marker short-circuit that returns before the DB provider is ever invoked; the fence-completion rule that clears the marker only for a settled abandonment or proven-terminal no-op and leaves it armed for every recovery-required outcome and every error, with `MarkerClearAfterCompletion` carrying the already-settled outcome; the CAS retry bounded by the coordinator's shared `MAX_CAS_RETRY_ATTEMPTS`, settling on a competitor's terminal status rather than overwriting it; and the deliberate false-negative tradeoff whereby a missing or `NeverSeen` target forces conservative strong recovery that may abandon unrelated live scopes, because attribution safety outranks preserving potentially valid evidence) - `context/cli/mutation-scope-runtime.md` (the crate-visible mutation-trace runtime seam and the lifecycle contract every current or future harness adapter must uphold, recorded by the `mutation-scope-runtime-integration` plan: the ten `pub(crate)` re-exports in `runtime/mod.rs` (`coordinate`, `RuntimeBoundary`, `StartProvenance`, `CoordinateOutcome`, `CoordinateError`, `ExternalTaintOperation`, `abandon_scope`, `AbandonScopeOutcome`, `AbandonRecoveryReason`, `AbandonScopeError`), with `ExternalTaintOperation` riding through `coordinator.rs`'s own `pub use` because `CoordinateError::ExternalTaintMarker` carries it — so the type becomes crate-visible without `protected_worktree` becoming a public module — while every `mod` declaration stays private and `ProtectedWorktree`/`ProtectedWorktreeError`/`WORKTREE_LOCK_TIMEOUT`/`reconcile_explicit` stay internal, kept clippy-clean by `#[allow(unused_imports)]` on the re-exports per the `services/style.rs` precedent rather than a placeholder consumer; and the adapter obligations themselves — a scope is one independently mutation-capable execution so a concurrent main agent and subagent need distinct `ScopeId`s, `Start`/`Advance`/`Close` semantics including that only `Start` may carry the optional `StartProvenance`, registered after scope registration and before the protocol commits and never inside `ProtocolState` or the CAS transition, with that registration conditional on the `ScopeState` `register_scope` returns so a provenance row may only be created while the scope is `NeverSeen` (an admission-time snapshot that a post-admission replay cannot backfill) while an existing row is still validated on every provenance-carrying `Start`, that a failed tool still requires `Advance` (the boundary is an observation, not a successful edit) and that a `ScopeId` is never reused after a terminal status (the `NeverSeen` guard silently refuses to reactivate it), `abandon_scope()` requiring positive staleness evidence and never inferring it from `ActorKind`, the `abandon` → `coordinate(Start(successor))` sequence with what each outcome implies for it including that a failed abandonment must never be treated as a safely started successor, that abandonment is not a `RuntimeBoundary` and needs no Quint change, the D1 tradeoff whereby a missing or `NeverSeen` target deliberately forces conservative strong recovery that may invalidate unrelated live scopes because attribution safety outranks preserving potentially valid evidence, and the attribution boundary that `AiExclusive(scope)` means scope exclusivity only and is not standalone proof that no human edited the worktree; a generic `sce hooks mutation-scope` ingress now drives the seam, joined by current Claude Code and Codex adapter drivers (both registered by `sce setup` and reachable) and an OpenCode adapter driver (full lifecycle + recovery, now generated as the final `sce-mutation-scope.ts` plugin and registered by `sce setup`, so reachable by a real session) — a Pi adapter driver now also exists (`cli/src/services/hooks/pi_mutation_scope/`, hidden `sce hooks pi-mutation-scope`) and is driven by the canonical generated Pi extension in ordinary Pi sessions; its human `user_bash` path uses the separate guarded external-mutation supervisor) - `context/cli/mutation-scope-hook-ingress.md` (the one harness-neutral CLI ingress that drives the mutation-scope runtime, in `cli/src/services/hooks/mutation_scope.rs`, built by the `mutation-scope-hook-ingress` plan: the hidden `sce hooks mutation-scope` command routing through the normal `cli_schema::HooksSubcommand::MutationScope` → `convert_hooks_subcommand_request` → `services::hooks::HookSubcommand::MutationScope` → `run_hooks_subcommand_in_repo` stack, reading one normalized JSON lifecycle object from STDIN; the strict `parse_mutation_scope_payload` contract supporting exactly `start`/`advance`/`close`/`flush`/`abandon` with a local `MutationScopePayload` transport enum, `claude_code`/`codex`/`opencode`/`pi` actor mapping, non-blank `scope_id`/`event_id`, exact per-operation key sets, an optional `provenance` object accepted on `start` only (a required non-blank `session_id`, an optional `model_id` where an absent key and an explicit `null` both mean no model, and no other key), and a dedicated hard rejection for any `worktree_id` key; the operation mapping to `RuntimeBoundary::Start`/`Advance`/`Close`/`Flush` through `coordinate()` or a direct `abandon_scope()` call, forwarding `ScopeId`/`EventId`/`ActorKind` and `start`'s optional `StartProvenance` verbatim because `EventId` equality is the runtime replay/idempotency key and provenance values arrive already canonical; identity ownership — the adapter owns `scope_id`/`event_id`/`actor_kind`, SCE owns `worktree_id`/Git tree identities/revisions/attempt IDs, and worktree identity is derived only by the runtime from the invoking checkout; the lazy `FnOnce` DB provider reusing `open_agent_trace_db_for_hook_runtime` so DB acquisition stays inside the runtime's protected-worktree ordering; the non-fail-open error classification by durable completion — malformed payload or any pre-completion `CoordinateError`/`AbandonScopeError` → `CliError`/non-zero, while `CoordinateError::MarkerClearAfterCommit` and `AbandonScopeError::MarkerClearAfterCompletion` are treated as durable success with empty stdout, the marker-cleanup failure logged via `sce.hooks.mutation_scope.marker_clear_after_durable_completion`, and the transition not retried; the empty-stdout success contract; abandonment ownership keeping no-snapshot semantics; and the generic-ingress vs harness-adapter boundary — a Claude Code adapter driver (`cli/src/services/hooks/claude_mutation_scope/`) and a Codex adapter driver (`cli/src/services/hooks/codex_mutation_scope/`, hidden `sce hooks codex-mutation-scope`) now exist and are registered by `sce setup`, both consuming this seam's own `pub(crate)` in-process entrypoint; an OpenCode adapter driver (`cli/src/services/hooks/opencode_mutation_scope/`, hidden `sce hooks opencode-mutation-scope`) also consumes it with a full `Start`/`Close`/`Abandon` lifecycle and checkout-local recovery state, and is now driven in production by the generated `sce-mutation-scope.ts` plugin installed last by `sce setup`; a Pi adapter driver (`cli/src/services/hooks/pi_mutation_scope/`, hidden `sce hooks pi-mutation-scope`) also consumes it now, with `(session-id, tool-call-id)`→`ScopeId` derivation and getppid()/`/proc`-start-time-based stale-process recovery; the canonical generated Pi extension drives it in ordinary sessions — see `context/cli/pi-mutation-scope-integration.md`) - `context/cli/claude-mutation-scope-integration.md` (the first concrete harness lifecycle adapter, `cli/src/services/hooks/claude_mutation_scope/`, hidden command `sce hooks claude-mutation-scope`, built by the `claude-mutation-scope-integration` plan: one independently mutation-capable Claude tool execution = one SCE mutation `ScopeId` (a session/prompt/main agent/subagent is never a scope); the `classify_tool` table (mutation-capable including unknown names, read-only `Read`/`Glob`/`Grep`/`WebFetch`/`WebSearch`/`AskUserQuestion`, `Agent` = delegation) plus the model-only `is_explicit_background_shell` predicate; the length-prefixed hash-free `cc-tool-v1|n=|s=..|a=..|t=..` `ScopeId` keyed on a monotonic checkout-local `attempt_seq` (never reused after terminal) with deterministic `|start` / `|close` `EventId`s; the `/sce/claude-mutation-scope-state.json` bookkeeping store (never attribution evidence, never synced) with its own separate lock never held across a seam call; `PreToolUse` write-ahead `pending_start` → seam `start` → `active` and its fail-closed Claude `permissionDecision: "deny"` on any failure (never `allow`, detail logged via `sce.hooks.claude_mutation_scope.pre_tool_use_fail_closed`); `PostToolUse`/`PostToolUseFailure` → `close`, with `pending_start`+terminal → abandon-not-late-start (D11) and failed-`close` → abandon-not-replay (D12); the abandonment cleanup signals (`PermissionDenied`, `Stop`/`StopFailure`, `UserPromptSubmit`, `SubagentStop`, `SessionEnd`, best-effort `WorktreeRemove` — the last two not observed to fire on Claude Code `2.1.258`); the `recovery_pending` barrier that denies new mutation-capable `PreToolUse` until quiescent then runs one seam `flush`; raw hook `cwd` (or `worktree_path` for `WorktreeRemove`) as authoritative repository root with no adapter-constructed `WorktreeId`; `run_in_background = true` handling — native Claude background Bash is operationally supported but unsupported for mutation attribution (allowed in `PreToolUse` before repository resolution with no mutation scope, adapter attempt, or lifecycle tracking, per the effective decision `context/decisions/2026-10-02-claude-background-bash-untracked.md`), while background `PowerShell` stays denied — and the separate self-detaching-descendant unsupported boundary (D20, with T04's Git-observable evidence — full detail split out to [`claude-mutation-scope-background-execution.md`](cli/claude-mutation-scope-background-execution.md) for the repository's per-file line budget); the ten unmatched `sce setup` registrations; the strict `claude_mutation_scope → hooks::mutation_scope → mutation_trace::runtime` dependency direction through the single `run_mutation_scope_from_payload` seam import (T05); admission-time exact model-state snapshot into `ScopeProvenance`; real Git/Agent Trace persistence coverage shared with the Codex path; and, added by the `doctor-mutation-scope-fix` plan's T04, a third `pending_abandon` attempt phase durably persisted before any abandon seam call — established evidence that abandonment has already been decided, distinct from "may still be running" — with every attempt a broad cleanup signal (`Stop`/`UserPromptSubmit`/`SubagentStop`/`SessionEnd`/`WorktreeRemove`) matches now marked `pending_abandon` in one write before any of their seam calls run, so one attempt's seam failure can never leave a batch sibling without its own retryable evidence; health classification detail split out to [`claude-mutation-scope-health.md`](cli/claude-mutation-scope-health.md)) - `context/cli/claude-mutation-scope-background-execution.md` (native Claude background Bash is operationally supported but unsupported for mutation attribution: `Bash(run_in_background=true)` is allowed in `PreToolUse` with no mutation scope, adapter attempt, lifecycle tracking, or attribution guarantee — effective record `context/decisions/2026-10-02-claude-background-bash-untracked.md` — its later terminal hooks are no-ops, and the accepted A/B contamination consequence is pinned by a regression; background `PowerShell` stays denied with the existing deny text; the T01 `2.1.284` probe result with lifecycle completion still unproven; and the separate self-detaching-descendant unsupported boundary D20 — Git-observable evidence that a foreground `setsid` command's detached descendant can still mutate the repository after `PostToolUse` returns — detail split out of `claude-mutation-scope-integration.md` for the repository's per-file line budget) - `context/cli/claude-mutation-scope-health.md` (the `doctor-mutation-scope-health` plan's Claude task, extended by the `doctor-mutation-scope-fix` plan's T04: `classify_health` in `cli/src/services/hooks/claude_mutation_scope/health.rs` maps `recovery_pending`/`attempts` onto `healthy | recovering | blocked | invalid` — `recovery_pending == false` is `healthy` regardless of live attempts; `recovery_pending == true` with `attempts` empty is `recovering` (the barrier's own next-`PreToolUse` flush clears it); `recovery_pending == true` with `attempts` non-empty is `blocked`, the exact incident shape this classifier exists to surface, since nothing in the ordinary hook lifecycle retries, removes, flushes, or clears from that shape; a read/parse failure is `invalid`. T04 adds a separate, `doctor`-invoked repair path (`assess_repairability`/`repair_blocked` in `health.rs`/`lifecycle.rs`, wired into `sce doctor --fix` by the same plan's T05) for the `blocked` shape: `AutoFixable` only when every currently persisted attempt is already `pending_abandon` (no established-abandon-intent `pending_start`/`active` attempt present); `repair_blocked` re-reads and re-proves that same condition inside one lock-protected read-only state transaction before releasing the lock and retrying each attempt's already-established seam `abandon` call independently, clearing the `recovery_pending` barrier only once no attempts remain — this classifier's own read-only behavior is unaffected by that repair path's existence) -- `context/cli/mutation-trace-ref-reconciliation.md` (the conservative per-worktree snapshot-ref reconciliation pass, built by the `mutation-cursor-ref-reconciliation` plan and wired by `mutation-cursor-ref-reconciliation-wiring`: the single lock-held algorithm `reconcile_with_held_lock` in `ref_reconciliation.rs`, the explicit production entrypoint `reconcile_explicit` in `ref_maintenance.rs` (one `WorktreeLock` acquisition, verified-existing Agent Trace DB, typed `ExplicitOutcome`), the documented 10 s concurrency contract, the `ref-maintenance.json` state file (past-only timestamps, skew normalization, atomic writes), the D4 verified-existing DB opener, the two-invariant durability model (per-worktree fail-closed local consistency vs repository-wide deletion safety, never a claim that every repository-wide root is pinned), and the deferred follow-ups (automatic out-of-band reclamation with its binding prerequisites, retired-worktree cleanup, evidence-based advice, historical retention); reconciliation runs only through `sce doctor --fix` and is never automatic) +- `context/cli/mutation-trace-ref-reconciliation.md` (the conservative per-worktree snapshot-ref reconciliation pass, built by the `mutation-cursor-ref-reconciliation` plan and wired by `mutation-cursor-ref-reconciliation-wiring`: the single lock-held algorithm `reconcile_with_held_lock` in `ref_reconciliation.rs`, the explicit production entrypoint `reconcile_explicit` in `ref_maintenance.rs` (one `WorktreeLock` acquisition, verified-existing Agent Trace DB, typed `ExplicitOutcome`), the documented 10 s concurrency contract, the `ref-maintenance.json` state file (past-only timestamps, skew normalization, atomic writes), the D4 verified-existing DB opener, the two-invariant durability model (per-worktree fail-closed local consistency vs repository-wide deletion safety, never a claim that every repository-wide root is pinned), a pointer to the missing-pin repair, and the deferred follow-ups (automatic out-of-band reclamation with its binding prerequisites, retired-worktree cleanup, evidence-based advice, historical retention); reconciliation runs only through `sce doctor --fix` and is never automatic) +- `context/cli/mutation-trace-missing-history-recovery.md` (the `sce doctor --fix` missing-pin repair `repair_missing_pins`, built by the `missing-history-recovery` plan: the missing-durable-history heuristic that detects missing history, not clone identity; three cases for the target worktree (prune events with absent trees and re-pin survivors, fail closed with `unrecoverable_cursor` on an absent cursor until one agent hook boundary records a present cursor, no action and no clone-identity claim when trees exist); repair runs only from `doctor --fix`, never from hooks or plain `sce doctor`; invariants I1–I5) - `context/cli/mutation-trace-ref-maintenance-state.md` (supporting contracts for snapshot-ref reconciliation: the per-worktree `/sce/ref-maintenance.json` state file with past-only timestamps, `FUTURE_SKEW_TOLERANCE` normalization on successful writes only, atomic staging-and-swap writes, the pure `evaluate_recommendation`, and the non-creating verified-existing Agent Trace DB opener with typed `ExistingRepositoryDbError` and per-DB-state guarantees) - `context/cli/mutation-scope-ref-advisory.md` (the advisory-only hook trigger in `cli/src/services/hooks/mutation_scope.rs` built by T03 of the `mutation-cursor-ref-reconciliation-wiring` plan: a lightweight `advise_if_due` check after a durably completed `Close`/`Flush`, logged and, when actionable, written as an `SCE:` line to an injectable stderr writer (best-effort delivery), with the per-worktree `ref-maintenance.json` state; no reconciliation, Git, or DB work on the hook path) - `context/cli/doctor-ref-reconciliation.md` (the `sce doctor --fix` caller of snapshot-ref reconciliation built by T04 of the `mutation-cursor-ref-reconciliation-wiring` plan: the `runtime::ref_doctor` facade, typed fix-result rows for completed/completed-with-persist-warning/failed/busy passes, and the read-only recommendation section/`ref_reconciliation` JSON object in plain `sce doctor`) diff --git a/context/glossary.md b/context/glossary.md index 10224a33f..1bd61e54c 100644 --- a/context/glossary.md +++ b/context/glossary.md @@ -36,14 +36,15 @@ - `atomic commit bypass mode`: The `/commit` mode selected when the exact first argument token is `oneshot` or `skip` (case-insensitive; the two aliases are behaviorally identical). It skips the staging-confirmation prompt, requires exactly one commit message covering all staged files with no split proposals, makes plan citations best-effort instead of blocking, and lets the command run exactly one `git commit`. Any other first token selects proposal-only regular mode. See `context/sce/atomic-commit-workflow.md`. - `split commit references`: The generated `sce-commit/references/atomic-commit.md`, `references/commit-message-style.md`, and `references/output.md` documents. The atomic reference owns staged-diff procedure, internal result branching, and commit boundaries; the style reference owns commit-message wording; and the output reference owns human-visible layouts. No `commit-contract.yaml` artifact or YAML result-contract section is generated. - `canonical OpenCode plugin registration source`: Shared Pkl-authored plugin-registration definition in `config/pkl/base/opencode.pkl`, re-exported from `config/pkl/renderers/common.pkl` as the canonical plugin list/path JSON consumed by OpenCode renderers before they emit generated `opencode.json` manifests; the current entries, in order, are `sce-bash-policy`, `sce-agent-trace`, and `sce-mutation-scope` (list order is load-bearing — see `sce-mutation-scope plugin`). -- `checkout identity`: Stable UUIDv7 identifier assigned to a cloned repository or linked Git worktree, stored in `/sce/checkout-id` (never committed) and resolved via `git rev-parse --git-dir`. The identity is created or reused by `sce setup` through `AgentTraceDbLifecycle::setup()` and also auto-created by hook runtime when `sce setup` has not been run. Checkout identity is now diagnostic metadata for repository-scoped Agent Trace storage; it does not select the active DB and is not stored on Agent Trace rows. Any pre-migration per-checkout DB files at `/sce/agent-trace-{checkout_id}.db` are never touched by SCE and are no longer inspectable via the CLI (the checkout-scoped DB surface was removed by the `retire-legacy-agent-trace-db` plan). See `context/cli/checkout-identity.md`. -- `repository identity`: Stable identity of a logical Git repository used to select the active repository-scoped Agent Trace DB path `/sce/repos//agent-trace.db` through the `agent_trace_storage` resolver. Resolved by `cli/src/services/repository_identity/` with precedence: explicit `agent_trace.repository_id` config value, then the URL of the configured Git remote (`agent_trace.repository_remote`, default `origin`), otherwise an actionable `.sce/config.json` error. Remote URLs canonicalize to a scheme-neutral, credential-free `host[:port]/path` form so equivalent SSH/SCP/HTTPS remotes converge, and the repository ID is `sha256("sce-repository-id-v1\0" + canonical_identity)` hex. Distinct from `checkout identity`, which stays per clone/worktree for diagnostics. Setup's named-remote preflight uses a strict lookup that returns `Ok(None)` only for a missing/empty URL, preserves other failures as runtime errors, and discards the URL, retaining only the remote name in technical failure sources. See `context/cli/repository-identity.md`. +- `checkout identity` (removed): The former per-clone/worktree UUIDv7 identifier stored in `/sce/checkout-id` was removed by the `remove-checkout-id` plan. SCE no longer creates or reads that file, `sce setup`/`sce doctor` no longer mention it, and there is no per-clone or per-checkout identity anywhere in current code; worktrees are identified only by Git topology (`resolve_worktree_id`: `main` for the primary worktree). It was diagnostic metadata only and never selected the active DB or appeared on Agent Trace rows. Any leftover `/sce/checkout-id` file or pre-migration `/sce/agent-trace-{checkout_id}.db` file is inert and never touched by SCE. An unresolvable worktree identity surfaces as `ReconcileError::WorktreeIdentity` (failure kind `worktree_identity`). See `context/cli/checkout-identity.md`. +- `missing durable history`: Durable mutation-trace roots (`cursor_tree`, event `before_tree`/`after_tree`) recorded in the repository-scoped Agent Trace DB for a worktree whose tree objects are absent from the current Git object store, typically after a sequential re-clone of the same remote inherits the deleted clone's `main` rows. Detected only by `sce doctor --fix` reconciliation as a heuristic: absence proves history this checkout cannot have kept, while a present tree proves nothing about clone identity. Absent event trees are pruned and survivors re-pinned; an absent cursor fails closed with `unrecoverable_cursor` until one agent hook boundary records a present cursor. See `context/cli/mutation-trace-missing-history-recovery.md`. +- `repository identity`: Stable identity of a logical Git repository used to select the active repository-scoped Agent Trace DB path `/sce/repos//agent-trace.db` through the `agent_trace_storage` resolver. Resolved by `cli/src/services/repository_identity/` with precedence: explicit `agent_trace.repository_id` config value, then the URL of the configured Git remote (`agent_trace.repository_remote`, default `origin`), otherwise an actionable `.sce/config.json` error. Remote URLs canonicalize to a scheme-neutral, credential-free `host[:port]/path` form so equivalent SSH/SCP/HTTPS remotes converge, and the repository ID is `sha256("sce-repository-id-v1\0" + canonical_identity)` hex. There is no per-clone counterpart: `checkout identity` was removed. Setup's named-remote preflight uses a strict lookup that returns `Ok(None)` only for a missing/empty URL, preserves other failures as runtime errors, and discards the URL, retaining only the remote name in technical failure sources. See `context/cli/repository-identity.md`. - `refresh single-flight guard`: Client-owned async coordination for Agent Trace control-plane authentication. Concurrent callers whose stored access token is expired, or whose request rejected the same token, serialize only refresh-and-save work, re-check credentials after acquiring the guard, and reuse the token saved by the first refresher; valid-token requests do not acquire the guard. The guard is held by the spawned task that performs renewal and save, so caller cancellation cannot release it mid-persistence; `wait_for_pending_refresh` acquires it as a completion barrier before `sce sync` returns. - `snapshot-ref reconciliation`: Explicit, per-worktree maintenance pass (`runtime::ref_maintenance::reconcile_explicit`, run only by `sce doctor --fix`) that deletes orphaned SCE snapshot pins under `refs/sce/mutation-cursor//` while retaining every tree in any worktree's durable roots; never automatic. See `context/cli/mutation-trace-ref-reconciliation.md`. - `ref-reconciliation advisory`: Lightweight hook-path check (`advise_if_due`) after a durably completed `Close`/`Flush` that may record and print a "reconciliation is recommended" reminder; it never reconciles, lists refs, opens the DB, or spawns Git. See `context/cli/mutation-scope-ref-advisory.md`. - `repository-scoped Agent Trace DB`: Active Agent Trace storage shape where one logical Git repository maps to `/sce/repos//agent-trace.db`. The current seam is `RepositoryAgentTraceDb = TursoDb` in `cli/src/services/agent_trace_db/repository.rs`, backed by the fresh multi-statement `001_repository_schema` baseline plus the additive `002_repository_source_instance_id` migration, with `repository_metadata` (`repository_id` plus `source_instance_id`) plus repository-level trace tables, no `checkout_id` columns, and typed repository-level insert helpers for diff traces, post-commit intersections, Agent Trace rows, messages, and parts. Hook runtime, Agent Trace setup/lifecycle, and `sce sync` resolve repository-scoped storage through `agent_trace_storage`. This is the sole Agent Trace DB adapter; the checkout-scoped adapter and former trace inspection surface were removed by the `retire-legacy-agent-trace-db` plan. Its nullable `agent_traces.commit_patch_id` column (migration `007_commit_patch_id`) stores the `commit patch ID`: an optional Git patch-equivalence fingerprint of the original commit, never proof of unique commit identity (see [agent-trace-db.md](sce/agent-trace-db.md)). - `source_instance_id`: Physical-database identity column on `repository_metadata`, independent of the logical `repository_id`. Added by the additive `002_repository_source_instance_id` migration (existing/placeholder rows default to an empty string); generated once per physical `agent-trace.db` by application code (`generate_source_instance_id()`, UUID v4 today) and validated with `is_valid_source_instance_id()` (non-empty once trimmed) — never generated in SQL and never derived from `repository_id`, remote URL, checkout ID, filesystem path, hostname, or user/workspace identity. `RepositoryAgentTraceDb::verify_or_initialize_repository_metadata` claims it with a concurrency-safe `UPDATE ... WHERE source_instance_id = ''`, so concurrent first opens of the same physical database converge on one winner and an already-valid value is never overwritten; the value stays stable across reopen and repeated `sce setup` runs. Two independently created databases for the same logical repository (for example two clones) get different `source_instance_id` values. See `context/sce/agent-trace-db.md`. -- `checkout registry` (removed): The central JSON registry at `/sce/checkout-registry.json` was removed in the `remove-checkout-registry` plan. The later trace inspection surface was also removed; current checkout IDs are diagnostic metadata only and are not discovered as command targets. Any pre-migration `/sce/agent-trace-*.db` files remain untouched. See `context/cli/checkout-identity.md`. +- `checkout registry` (removed): The central JSON registry at `/sce/checkout-registry.json` was removed in the `remove-checkout-registry` plan. The later trace inspection surface and checkout identity itself (`remove-checkout-id` plan) were also removed, so nothing is discovered as a per-checkout command target. Any pre-migration `/sce/agent-trace-*.db` files remain untouched. See `context/cli/checkout-identity.md`. - `generated OpenCode plugin registration contract`: Current generated-config contract where `config/.opencode/opencode.json` serializes the OpenCode `plugin` field from canonical Pkl sources for SCE-managed plugins only; the current registered paths, in order, are `./plugins/sce-bash-policy.ts`, `./plugins/sce-agent-trace.ts`, and `./plugins/sce-mutation-scope.ts`. `sce-mutation-scope` must stay last; the OpenCode config merge appends the generated SCE entries after arbitrary user plugins and `sce doctor` (`inspect_opencode_plugin_ordering_health`) flags a non-last position. Claude does not use an OpenCode-style plugin manifest; Claude bash-policy enforcement is registered through generated `.claude/settings.json` as a `PreToolUse` `Bash` command hook running `sce policy bash`. - `root Biome contract`: Repository-root formatting/linting contract owned by `biome.json`, currently scoped only to `npm/**` and the shared `config/lib/**` plugin package root with package-local `node_modules/**` excluded; the canonical execution path is the root Nix dev shell (`nix develop -c biome ...`). - `cli flake checks`: Check derivations in root `flake.nix` (`checks..cli-tests`, `cli-clippy`, `cli-fmt`), plus `pkl-generated`, `workflow-actionlint`, split npm/config-lib JS checks, and Linux-only Flatpak checks; invoked via `nix flake check` at repo root. @@ -116,7 +117,7 @@ - `setup required-hook install orchestration`: Setup-service flow in `cli/src/services/setup/mod.rs` (`install_required_git_hooks`, backed by the rename-injectable `install_required_git_hooks_with_rename`) that resolves repository root + effective hooks directory via git truth, then for each hook computes the bytes to stage with the `setup hook-merge seam` (`hook_merge::merge_or_create_hook`) instead of writing the canonical asset verbatim, reports deterministic per-hook outcomes (`Installed`, `Updated`, `Skipped`) against that merged content plus the executable bit, enforces executable permissions, sets `RequiredHookInstallResult.unreachable_block_advisory` (rendered as a named advisory line) when an appended block would be unreachable, and uses the `setup atomic-swap` policy (see `setup atomic-swap`) — staged content is renamed directly over an existing hook without unlinking it first — with deterministic recovery guidance on swap failure. - `setup hooks CLI mode`: `sce setup` behavior activated by `--hooks` (with optional `--repo `), supporting both hooks-only runs and composable target+hooks runs in one invocation; implemented through `cli/src/services/setup/command.rs` + `cli/src/services/setup/mod.rs`, enforces deterministic compatibility validation (`--repo` requires `--hooks`; target flags remain mutually exclusive), and emits stable setup/hook status output. - `setup repo gate`: Preflight check in `cli/src/services/setup/command.rs` that calls `cli/src/services/setup/mod.rs` (`ensure_git_repository`) and validates the effective named-remote URL before any setup writes begin; all `sce setup` modes (config-only, hooks-only, combined, interactive, and `--bootstrap-context`) require a Git repository and configured remote. The setup-owned `GitRepositoryResolutionError` maps only a positively identified non-Git directory to `setup.not_git_repository`, while missing/empty configured remote URLs map to `setup.not_git_remote`; nonexistent, inaccessible, process, malformed-output, and other Git/remote-lookup failures map to `general.unexpected_failure`, with technical sources preserved through the CLI user-error boundary. -- `setup local bootstrap`: Pre-install setup bootstrap behavior now owned by lifecycle providers: `ConfigLifecycle::setup` creates missing `.sce/config.json` with the canonical versioned schema-only payload (`{"$schema": "https://sce.crocoder.dev/v/config.json"}`, using the CLI release version), `LocalDbLifecycle::setup` initializes the canonical local DB via `LocalDb::new()`, and `AgentTraceDbLifecycle::setup` creates/reuses checkout identity, resolves repository identity, initializes the repository-scoped Agent Trace DB via `agent_trace_storage`, and records repository ID, checkout ID, and `database_path`; the setup command aggregates these calls before config/hooks dispatch across all normal setup modes after context baseline bootstrap. +- `setup local bootstrap`: Pre-install setup bootstrap behavior now owned by lifecycle providers: `ConfigLifecycle::setup` creates missing `.sce/config.json` with the canonical versioned schema-only payload (`{"$schema": "https://sce.crocoder.dev/v/config.json"}`, using the CLI release version), `LocalDbLifecycle::setup` initializes the canonical local DB via `LocalDb::new()`, and `AgentTraceDbLifecycle::setup` resolves repository identity, initializes the repository-scoped Agent Trace DB via `agent_trace_storage`, and records repository ID and `database_path`; the setup command aggregates these calls before config/hooks dispatch across all normal setup modes after context baseline bootstrap. - `setup context baseline bootstrap`: Additive durable-context tree bootstrap in `cli/src/services/setup/mod.rs` (`bootstrap_context_baseline`) that create-if-missing writes neutral baseline Markdown files, working directories, and `context/tmp/.gitignore` via `RepoPaths` accessors. `sce setup --bootstrap-context` is the dedicated context-only mode and must be used alone; every normal successful setup path also ensures the same baseline after the Git gate and before lifecycle/config install work without overwriting existing content. - `CLI redaction-safe diagnostics contract`: baseline security behavior implemented via `cli/src/services/security.rs` (`redact_sensitive_text`) and applied to app-level errors, setup git-diagnostic surfacing, and observability output sinks so common secret-bearing token forms are masked before emission. - `setup directory write-permission probe`: deterministic pre-write guard implemented in `cli/src/services/security.rs` (`ensure_directory_is_writable`) and used by setup install/hook flows to fail fast with actionable remediation when target directories are not writable. @@ -135,7 +136,7 @@ - `app startup phases`: Current `cli/src/app.rs` execution model that separates dependency checking, startup-context construction, runtime initialization, command parse/execute, and output rendering into named helpers while preserving the CLI's existing exit-code, stderr-diagnostic, and degraded-startup behavior; output rendering and execution-phase logging helpers live in `cli/src/services/app_support.rs`. - `RunOutcome`: Generic final render payload in `cli/src/services/app_support.rs` (`RunOutcome`) carrying a command result, optional startup diagnostic, and optional logger implementing the logger trait boundary. Production construction in `cli/src/app.rs` uses the concrete observability logger, while rendering is not hardcoded to that production type. - `CommandRegistry`: Static command-name catalog in `cli/src/services/command_registry.rs`; populated by `build_default_registry()` and carried by `AppRuntime` during command dispatch. It exposes deterministic command-name membership for the current top-level command catalog (`help`, `auth`, `config`, `setup`, `doctor`, `hooks`, `policy`, `sync`, `version`, and `completion`) while actual parsed command payloads are represented by the `RuntimeCommand` enum rather than zero-arg boxed constructors. -- `ServiceLifecycle`: Compile-safe lifecycle trait seam in `cli/src/services/lifecycle.rs` with default no-op generic `diagnose`, `fix`, and `setup` methods over `C: HasRepoRoot`; it exposes lifecycle-owned health, fix, and setup result types, while doctor/setup adapt those records at orchestration boundaries before rendering command-owned output. The hooks service has `HooksLifecycle` for hook rollout diagnosis/fix/setup, the config service has `ConfigLifecycle` for global/repo-local config validation plus repo-local config bootstrap, local_db has `LocalDbLifecycle` for canonical local DB path health/bootstrap/setup, auth_db has `AuthDbLifecycle` for canonical auth DB path health/bootstrap/setup, and agent_trace_db has `AgentTraceDbLifecycle` for repository identity resolution, checkout identity setup for diagnostics, setup-time repository-scoped Agent Trace DB initialization, and repository Agent Trace DB path health/bootstrap, returning an actionable "requires a Git repository" diagnostic outside repository context (the former global parent fallback was removed by the `retire-legacy-agent-trace-db` plan). Doctor runtime aggregates the static provider catalog for `diagnose` and `fix`; setup command aggregates providers for `setup` in order (config → local_db → auth_db → agent_trace_db → hooks when requested). +- `ServiceLifecycle`: Compile-safe lifecycle trait seam in `cli/src/services/lifecycle.rs` with default no-op generic `diagnose`, `fix`, and `setup` methods over `C: HasRepoRoot`; it exposes lifecycle-owned health, fix, and setup result types, while doctor/setup adapt those records at orchestration boundaries before rendering command-owned output. The hooks service has `HooksLifecycle` for hook rollout diagnosis/fix/setup, the config service has `ConfigLifecycle` for global/repo-local config validation plus repo-local config bootstrap, local_db has `LocalDbLifecycle` for canonical local DB path health/bootstrap/setup, auth_db has `AuthDbLifecycle` for canonical auth DB path health/bootstrap/setup, and agent_trace_db has `AgentTraceDbLifecycle` for repository identity resolution, setup-time repository-scoped Agent Trace DB initialization, and repository Agent Trace DB path health/bootstrap, returning an actionable "requires a Git repository" diagnostic outside repository context (the former global parent fallback was removed by the `retire-legacy-agent-trace-db` plan). Doctor runtime aggregates the static provider catalog for `diagnose` and `fix`; setup command aggregates providers for `setup` in order (config → local_db → auth_db → agent_trace_db → hooks when requested). - `lifecycle provider catalog`: Shared factory in `cli/src/services/lifecycle.rs` (`lifecycle_providers(include_hooks)`) that returns static `LifecycleProvider` enum values in deterministic config → local_db → auth_db → agent_trace_db → hooks order, used by doctor with hooks included and by setup with hooks included only when requested. The enum owns concrete-provider dispatch for `id`, `diagnose`, `fix`, and `setup` without boxed provider trait objects or `&dyn HasRepoRoot` lifecycle context erasure. - `sce config command surface`: Implemented top-level CLI command routed by `cli/src/app.rs` to `cli/src/services/config/mod.rs` (with schema/file-parsing delegated to `cli/src/services/config/schema.rs`, runtime precedence resolution delegated to `cli/src/services/config/resolver.rs`, and text/JSON output construction delegated to `cli/src/services/config/render.rs`), exposing `show`, `validate`, and `--help` for deterministic runtime config inspection and validation; `show` reports resolved flat logging observability values with provenance, while `validate` reports pass/fail plus validation issues and warnings only. - `sce version command surface`: Implemented top-level CLI command routed by `cli/src/app.rs` to `cli/src/services/version/mod.rs` plus `cli/src/services/version/command.rs`, exposing deterministic runtime identification output in text form by default and JSON form via `--format json`. @@ -189,7 +190,7 @@ - `agent trace commit-msg co-author policy`: Current contract in `cli/src/services/hooks/mod.rs` (`apply_commit_msg_coauthor_policy`) that applies exactly one canonical trailer (`Co-authored-by: SCE `) only when attribution hooks are enabled, SCE is not disabled, and the staged-diff AI-overlap preflight confirms AI/editor evidence (`StagedDiffAiOverlapResult::Overlap`); `NoOverlap` and `Error` both suppress the trailer, with `Error` logged via `sce.hooks.commit_msg.ai_overlap_error`; duplicate canonical trailers are deduped idempotently. - `local DB migration contract`: `cli/src/services/local_db/mod.rs` delegates migration execution to `TursoDb` through the `DbSpec::migrations()` contract. The current `LocalDbSpec` migration list is empty, so `LocalDb::new()` opens/creates the canonical local DB without creating local tables. - `hook no-op baseline`: Current `cli/src/services/hooks/mod.rs` runtime posture where `pre-commit` and `post-rewrite` return deterministic no-op status text, `commit-msg` is a gated mutating path, `post-commit` persists intersections and built Agent Trace payloads without post-commit file artifacts, `diff-trace` validates STDIN payloads, resolves Claude `model_id` with `direct > exact transcript > exact session/agent state > bridge-derived chain state > NULL` precedence, persists direct `tool_version`, applies stored `session_id` prefixes (`oc_`/`cc_`/`pi_`), and inserts DB-only AgentTraceDb rows, `conversation-trace` is the active message/part intake path, and `codex` (`cli/src/services/hooks/codex/`) classifies its own raw hook JSON into four supported dispatch arms — `UserPromptSubmit` and `Stop` capture real conversation evidence, `PreToolUse(Bash)` delegates to the existing Bash policy engine, `PostToolUse(apply_patch)` parses/normalizes/persists a `diff_traces` row using event-scoped synthetic line identities — with every other event/tool combination (including `PreToolUse(apply_patch)`) and malformed STDIN failing open as a no-op. `session-model` is no longer a supported hook route. -- `sce doctor` operator-health contract: `cli/src/services/doctor/mod.rs` is the stable doctor entrypoint, with focused `doctor/{inspect,render,fixes,types}.rs` submodules implementing the current approved operator-health surface in `context/sce/agent-trace-hook-doctor.md`: `sce doctor --fix` selects repair intent, Agent Trace DB discovery is repository-scoped only (the checkout-scoped `sce trace --legacy` surface was removed by the `retire-legacy-agent-trace-db` plan), and output exposes deterministic doctor mode, readiness, stable problem taxonomy/fixability fields, checkout/database records, and fix-result records. The runtime validates state-root resolution, global and repo-local `sce/config.json` readability/schema health, local DB and repository-scoped Agent Trace DB path/health, DB-parent readiness barriers, git availability, non-repo vs bare-repo targeting failures, effective hook-path source resolution, required hook presence/executable/content drift against canonical embedded hook assets, and repo-root installed OpenCode, Claude, Pi, plus Codex integration content health. Human text mode uses the approved sectioned layout (`Environment`, `Configuration` with checkout identity plus repository-scoped Agent Trace DB rows when available, `Repository`, `Git Hooks`, `Integrations`), `SCE doctor diagnose` / `SCE doctor fix` headers, bracketed `[PASS]`/`[FAIL]`/`[MISS]` status tokens with shared-style green/red colorization when enabled, simplified `label (path)` row formatting, top-level-only hook rows, and integration parent/child rows where missing files surface as `[MISS]`, mismatches/read failures as `[FAIL]`, and affected parent groups as `[FAIL]`. Agent Trace DB rows include repository ID, identity source, safe canonical identity, configured remote name, and never raw remote URLs. Current integration groups include `OpenCode plugins`, `OpenCode agents`, `OpenCode commands`, `OpenCode skills`, `ClaudeCode plugins`, `ClaudeCode commands`, `ClaudeCode skills`, `Pi prompts`, `Pi skills`, `Pi extensions`, `Codex skills`, and `Codex hooks`; Claude `settings.json` plus `hooks/**` belong to `ClaudeCode plugins`, including `.claude/hooks/run-sce-or-show-install-guidance.sh`, Pi `prompts/**` and `skills/**` map to the Pi groups, and Codex's `.agents/skills/**` plus `.codex/hooks.json`/`.codex/hooks/**` map to the Codex groups (the latter also carrying a Codex hook trust/review reminder when unhealthy). Fix mode reuses canonical setup hook installation for missing/stale/non-executable required hooks and missing hooks directories and can bootstrap canonical missing SCE-owned DB parent directories. +- `sce doctor` operator-health contract: `cli/src/services/doctor/mod.rs` is the stable doctor entrypoint, with focused `doctor/{inspect,render,fixes,types}.rs` submodules implementing the current approved operator-health surface in `context/sce/agent-trace-hook-doctor.md`: `sce doctor --fix` selects repair intent, Agent Trace DB discovery is repository-scoped only (the checkout-scoped `sce trace --legacy` surface was removed by the `retire-legacy-agent-trace-db` plan), and output exposes deterministic doctor mode, readiness, stable problem taxonomy/fixability fields, checkout/database records, and fix-result records. The runtime validates state-root resolution, global and repo-local `sce/config.json` readability/schema health, local DB and repository-scoped Agent Trace DB path/health, DB-parent readiness barriers, git availability, non-repo vs bare-repo targeting failures, effective hook-path source resolution, required hook presence/executable/content drift against canonical embedded hook assets, and repo-root installed OpenCode, Claude, Pi, plus Codex integration content health. Human text mode uses the approved sectioned layout (`Environment`, `Configuration` with repository-scoped Agent Trace DB rows when available, `Repository`, `Git Hooks`, `Integrations`), `SCE doctor diagnose` / `SCE doctor fix` headers, bracketed `[PASS]`/`[FAIL]`/`[MISS]` status tokens with shared-style green/red colorization when enabled, simplified `label (path)` row formatting, top-level-only hook rows, and integration parent/child rows where missing files surface as `[MISS]`, mismatches/read failures as `[FAIL]`, and affected parent groups as `[FAIL]`. Agent Trace DB rows include repository ID, identity source, safe canonical identity, configured remote name, and never raw remote URLs. Current integration groups include `OpenCode plugins`, `OpenCode agents`, `OpenCode commands`, `OpenCode skills`, `ClaudeCode plugins`, `ClaudeCode commands`, `ClaudeCode skills`, `Pi prompts`, `Pi skills`, `Pi extensions`, `Codex skills`, and `Codex hooks`; Claude `settings.json` plus `hooks/**` belong to `ClaudeCode plugins`, including `.claude/hooks/run-sce-or-show-install-guidance.sh`, Pi `prompts/**` and `skills/**` map to the Pi groups, and Codex's `.agents/skills/**` plus `.codex/hooks.json`/`.codex/hooks/**` map to the Codex groups (the latter also carrying a Codex hook trust/review reminder when unhealthy). Fix mode reuses canonical setup hook installation for missing/stale/non-executable required hooks and missing hooks directories and can bootstrap canonical missing SCE-owned DB parent directories. - `compact doctor text report`: Current human-readable `sce doctor` rendering collapses healthy state, repository, and typed integration-area checks into the `Environment`/`Repository`/`Integrations` hierarchy, suppresses healthy paths and identity metadata, and uses `--format json` as the full-detail route. The canonical current contract is [doctor human text](sce/doctor-human-text-contract.md). - `cli warnings-denied lint policy`: `cli/Cargo.toml` sets `warnings = "deny"`, so plain `cargo clippy --manifest-path cli/Cargo.toml` already fails on warnings without needing an extra `-- -D warnings` tail. - `agent trace local DB schema migration contract`: Retired `apply_core_schema_migrations` behavior removed from the current runtime during `agent-trace-removal-and-hook-noop-reset` T01; the local DB baseline is now file open/create only. diff --git a/context/overview.md b/context/overview.md index 358f57f86..50eec1859 100644 --- a/context/overview.md +++ b/context/overview.md @@ -88,7 +88,7 @@ The same config resolver now also owns the attribution-hooks gate used by local The config service split now includes `cli/src/services/config/resolver.rs` as the focused owner for config-file discovery, file-layer merging, env/flag/default precedence, auth-key resolution, observability resolution, attribution-hooks resolution, and default-discovered invalid-file degradation; `cli/src/services/config/mod.rs` remains the facade/rendering orchestration surface while preserving existing `services::config` imports. Generated config now includes repo-local OpenCode plugin assets: `sce-bash-policy.ts` plus `sce-agent-trace.ts` are emitted under `config/.opencode/plugins/`; the OpenCode agent-trace plugin extracts `{ sessionID, diff, time, model_id }` from user `message.updated` events with diffs, tracks per-session OpenCode client version from `session.created`/`session.updated`, and sends payloads to `sce hooks diff-trace` with `tool_name="opencode"` plus optional `tool_version`. Claude generated config registers `SessionStart` and `PostModelSwitch` for local model-state intake through `sce hooks claude-model-state`, while supported `PostToolUse Write|Edit|MultiEdit|NotebookEdit` events remain routed directly to `sce hooks diff-trace`; it does not call the retired `sce hooks session-model` route. Rust handles extraction, validation, and persistence without a TypeScript intermediary; the former `config/.claude/plugins/sce-agent-trace.ts` Bun runtime was removed in T07 of the `claude-rust-diff-trace` plan. The Rust hook validates required fields, resolves Claude `model_id` with `direct > exact transcript > exact session/agent state > bridge-derived chain state > NULL` precedence using direct metadata, matching `transcript_path`/`tool_use_id` JSONL fallback, then one exact local state lookup while keeping `tool_version` direct (without restoring `session_models`), and persists tool-prefixed `session_id` values (`oc*`for OpenCode,`cc*`for Claude,`pi*`for Pi),`model_id`, `tool_name`, and nullable `tool_version`into`diff_traces`through AgentTraceDb. Bash-policy now delegates OpenCode enforcement to the Rust`sce policy bash`command: the generated OpenCode plugin at`config/.opencode/plugins/sce-bash-policy.ts`is a thin wrapper that calls`sce policy bash --input normalized --output json`via`spawnSync`and throws on deny decisions; it no longer contains independent TypeScript policy logic. The former`bash-policy/runtime.ts`TypeScript runtime has been removed. Preset... Claude bash-policy enforcement is also generated through`.claude/settings.json`as a`PreToolUse` `Bash`command hook running`sce policy bash`, so Claude and OpenCode both delegate to the Rust policy evaluator without a Claude TypeScript runtime. Pi bash-policy enforcement is delegated the same way through a project-local Pi extension (`config/lib/pi-plugin/sce-pi-extension.ts`, emitted to `config/.pi/extensions/sce/index.ts`) whose `tool_call`handler blocks denied bash commands via`sce policy bash`and fails open when the policy check cannot run (see`context/sce/pi-extension-runtime.md`). -Local database bootstrap is now owned by `LocalDbLifecycle::setup`and`AgentTraceDbLifecycle::setup`aggregated by the setup command. Agent Trace lifecycle setup creates/reuses the current checkout ID for diagnostics and creates or migrates the repository-scoped`/sce/repos//agent-trace.db`; hook runtime uses the same repository-storage identity/path resolution without running migrations, and missing or stale schema causes hook paths to fail open with the existing`Run 'sce setup'.`guidance. Doctor validates the repository-scoped DB path/health and can bootstrap missing parent directories; outside a Git repository it reports an actionable "requires a Git repository" diagnostic instead of probing a sentinel path.`sce sync`is fully implemented: it resolves repository-scoped storage, authenticates against the control plane with stored WorkOS credentials, fetches authoritative cursors once, synchronizes three of the four local Agent Trace capture streams (`messages`, `parts`, `agent_traces`) concurrently while preserving sequential batches within each stream, leaves `diff_traces` local, and renders the documented concise text/JSON output (see`context/cli/sync-command.md`). The former `sce trace` command group and its database inspection surfaces are unavailable. +Local database bootstrap is now owned by `LocalDbLifecycle::setup`and`AgentTraceDbLifecycle::setup`aggregated by the setup command. Agent Trace lifecycle setup creates or migrates the repository-scoped`/sce/repos//agent-trace.db`; hook runtime uses the same repository-storage identity/path resolution without running migrations, and missing or stale schema causes hook paths to fail open with the existing`Run 'sce setup'.`guidance. Doctor validates the repository-scoped DB path/health and can bootstrap missing parent directories; outside a Git repository it reports an actionable "requires a Git repository" diagnostic instead of probing a sentinel path.`sce sync`is fully implemented: it resolves repository-scoped storage, authenticates against the control plane with stored WorkOS credentials, fetches authoritative cursors once, synchronizes three of the four local Agent Trace capture streams (`messages`, `parts`, `agent_traces`) concurrently while preserving sequential batches within each stream, leaves `diff_traces` local, and renders the documented concise text/JSON output (see`context/cli/sync-command.md`). The former `sce trace` command group and its database inspection surfaces are unavailable. The repository-root flake (`flake.nix`) applies a Rust overlay-backed stable toolchain pinned to `1.95.0`(with`rustfmt`and`clippy`), reads package/check version from the repo-root `.version`file, and builds`packages.sce`through a Crane`buildDepsOnly`+`buildPackage`pipeline. One deterministic pre-Cargo Nix derivation invokes the shared generated-input producer and supplies its validated`SCE_CLI_GENERATED_INPUT_DIR`store path to native, release, test, and Clippy Cargo derivations. Pkl is absent from those Cargo environments; dependency-only and format derivations do not receive the handoff, so canonical generation changes invalidate compiling outputs without invalidating dependency artifacts or formatting.`cli-tests`, `cli-clippy`, and `cli-fmt`remain Crane-backed check derivations. The root flake splits native and release outputs:`packages.sce`and`packages.default` build the **native** development binary (`scePackage`), while `packages.sce-release` builds the release binary (`sceReleasePackage`: static musl on Linux, native on Darwin). So `nix build .#sce`/`.#default`, `nix run . -- --help`, `nix run .#sce -- --help`, and `nix profile install github:crocoder-dev/shared-context-engineering`target the native binary, and`nix build .#sce-release`/`nix run .#sce-release -- ...`(plus`nix run .#release-artifacts`, which builds `.#sce-release`) target the release binary. On Linux the native and release outputs are distinct store paths, and the release output passes the native portability audit. `packages..ci-checks`is the explicit long-running validation tier:`nix build .#ci-checks`builds the`.#sce-release`package and, on Linux, audits the real release binary for forbidden`/nix/store/`references, so the expensive work stays out of`nix flake check`(which never builds`.#sce-release`). Git-commit embedding is **release-only**: `SCE_GIT_COMMIT`is injected via a`releaseCommitArgs` fragment applied only to the release derivations (`scePackageMusl`on Linux,`sceReleasePackageNative`on Darwin), not to`commonCargoArgs`. So native `.#sce`/`.#default`and every`nix flake check` derivation (`cli-tests`, `cli-clippy`, `cli-fmt`) build without the commit in their inputs and stay cache-reusable across commits (native `sce version`reports`unknown`), while `.#sce-release`still reports the real commit via`sce version`. `cli/build.rs` `emit_git_commit`emits`SCE_GIT_COMMIT`only when the env var is explicitly set — no`git rev-parse`fallback and no`.git/HEAD`/`.git/packed-refs`rerun watches. On Darwin the release now uses a distinct native-toolchain derivation (native toolchain + commit), so it diverges from`.#sce`to carry the commit while native stays commit-independent. diff --git a/context/plans/missing-history-recovery.md b/context/plans/missing-history-recovery.md new file mode 100644 index 000000000..3291aa14f --- /dev/null +++ b/context/plans/missing-history-recovery.md @@ -0,0 +1,173 @@ +# Plan: missing-history-recovery + +## Change summary + +Make sequential re-clones of the same remote recoverable without any per-clone +identity. The repository-scoped Agent Trace DB is keyed by remote identity and +the primary worktree is always `main`, so a fresh clone inherits the deleted +clone's `main` cursor row and `mutation_trace_events` while their snapshot trees +and `refs/sce/mutation-cursor/main/*` pins stayed in the deleted `.git` +(observed 2026-10-01: 1017 unpinned roots, 985 absent objects, every +`sce doctor --fix` failing with `missing_required_pins`). + +Detection is a heuristic: a durable tree that is absent from the current Git +object store is history this checkout cannot have produced or kept. It detects +missing history, not clone identity. A durable tree that does exist (for example +a pushed commit tree) is not evidence that the clone is unchanged. Repair runs +only from the explicit `sce doctor --fix` reconciliation pass; the agent hook +path never destructively resets history, and plain `sce doctor` keeps relying on +the existing reconciliation recommendation. + +The plan distinguishes three cases for the target worktree `W`: + +1. **Missing non-cursor trees:** prune every `W` event whose `before_tree` or + `after_tree` is absent (with its `mutation_trace_event_active_scopes` rows) + and re-pin the surviving unpinned roots. This repair already exists as + uncommitted work in progress in the working tree (`repair_missing_pins`, + `GitSnapshotService::{absent_trees, pin_trees}`, + `MutationTraceStore::{load_mutation_event_trees, prune_mutation_events}`, + new doctor counts and failure kinds, tests, context). The plan preserves it + and lands it as T01. +2. **Missing cursor tree:** the pass keeps failing closed with + `unrecoverable_cursor` and changes nothing. This state is transient: the next + agent hook boundary commits a new present cursor and turns the old cursor into + an event `before_tree`, which case 1 then prunes. No separate cursor-recovery + mechanism is built. +3. **Existing cursor tree:** no destructive action and no claim that the clone is + unchanged. Reconciliation continues under the stated heuristic limitation. + +## Acceptance criteria + +- [x] AC1: With a durable `main` cursor that exists and events that reference + absent trees, `sce doctor --fix` prunes exactly the events with an absent + `before_tree` or `after_tree` (and their active-scope rows), keeps every other + event, re-pins the surviving unpinned roots, and completes. + - Validate: `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml ref_reconciliation_prunes_events_whose_trees_are_absent_and_repins_the_rest` runs one test and it passes (confirm with `-- --list`). +- [x] AC2: With a durable `main` cursor tree that is absent from the object + store, `sce doctor --fix` fails with `unrecoverable_cursor`, prunes no event, + changes no pin, and its message tells the operator to let one agent tool run + in this checkout and then rerun `sce doctor --fix`. + - Validate: `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml ref_reconciliation_absent_cursor_tree_fails_closed_without_pruning` runs one test and it passes, and the `UnrecoverableCursor` display text contains `rerun \`sce doctor --fix\``. +- [x] AC3: No agent hook boundary prunes events, rewrites the cursor, or abandons + scopes because of absent trees. + - Validate: `rg -n "prune_mutation_events|repair_missing_pins" cli/src/services/hooks cli/src/services/mutation_trace/runtime/coordinator.rs` returns no matches. +- [x] AC4: No per-clone identity is introduced: no checkout or clone ID file, no new identity column or migration, and `resolve_worktree_id` still returns `main` for the primary worktree. + - Validate: `ls cli/migrations/agent-trace-repository/` gains no file from this plan's commits, and `rg -n "checkout_id|clone_id" cli/src` returns no matches. +- [x] AC5: Durable context describes the three cases and the heuristic limitation, + and no context file claims SCE currently tracks checkout or clone identity. + - Validate: `rg -in "creates/reuses checkout identity|Stable UUIDv7 identifier assigned" context/glossary.md` returns no matches, and `context/cli/mutation-trace-ref-reconciliation.md` states that an existing tree is not evidence of an unchanged clone. + +### Full validation + +- `nix flake check` + +### Context sync + +- `context/cli/mutation-trace-ref-reconciliation.md` +- `context/cli/doctor-ref-reconciliation.md` +- `context/cli/mutation-trace-runtime-coordinator.md` +- `context/glossary.md` +- `context/context-map.md` + +## Task context synchronization lifecycle + +Persist this field in every plan; this is durable plan state, not chat state: + +- **Task context synchronization:** every task carries `pending | synced | blocked`. + A completed task must be `synced` before another task can start or the plan can + finish. +- For `blocked`, record **Blocker**, **Required action**, and **Retry condition** + beside the status. Never infer `synced` from conversation history; write every + lifecycle transition to the plan file. + +## Constraints and non-goals + +- **In scope:** `cli/src/services/mutation_trace/runtime/{ref_reconciliation,ref_maintenance,ref_doctor,git_snapshot}.rs`, `cli/src/services/mutation_trace/store.rs`, `cli/src/services/doctor/{mod,render}.rs`, focused tests under `cli/src/services/mutation_trace/runtime/ref_maintenance/tests*` and the doctor test modules, and the context files listed under Context sync. +- **Out of scope:** recovering an absent cursor through a new cursor transition; plain `sce doctor` detection of absent trees; concurrent independent clones of the same remote sharing one DB; clone-qualified worktree IDs; any change to the `refs/sce/mutation-cursor//` namespace; the orphaned legacy-format worktree row (`01a09050-…`) and other non-target worktrees; the `AGENTS.md` Turso-shell update already in the working tree (a separate commit). +- **Constraints:** + - Preserve the `remove-checkout-id` architecture: no checkout ID, clone ID, persistent marker, or schema migration for identity. + - Destructive DB changes happen only in the explicit `doctor --fix` pass, under the held worktree lock, behind a revision CAS, and are idempotent across a crash between steps. + - The verified maintenance connection stays read-only; a writable connection is opened on the verified path only when a prune is required, after the read-only one is dropped. + - Code stays comment-free, matching the surrounding modules. +- **Non-goal:** proving clone identity. An existing tree is never treated as proof that history is local. + +## Invariants + +- I1: After a completed pass, every durable root of `W` (cursor and event trees) is present in the object store and pinned under `W`'s namespace. +- I2: An event is pruned only if one of its trees is absent; every event whose trees are all present survives. +- I3: The pass never changes `W`'s cursor; an absent cursor fails closed with nothing changed. +- I4: A failure at any step leaves either the pre-pass state or a state the next pass repairs (a prune committed before a failed re-pin leaves only present unpinned roots). +- I5: Stale-pin deletion keeps its repository-wide rule: a pin is deleted only if its tree is a durable root of no worktree. + +## Assumptions + +- The working-tree repair is landed as-is in T01, except for the `UnrecoverableCursor` guidance text and any fixes its own verification requires. +- An absent cursor is transient because every committed hook boundary replaces `cursor_tree` with the freshly captured, pinned tree (`coordinator.rs` commit path). + +## Task stack + +- [x] T01: `Repair missing pins and prune unrecoverable events in doctor --fix` (status:done) + - Task ID: T01 + - Scope: In — the uncommitted repair already in the working tree: `absent_trees`/`pin_trees`, `load_mutation_event_trees`/`prune_mutation_events`/public `load_worktree_state`, `repair_missing_pins` with `MaintenanceDb` path threading, new `ReconcileError` variants and failure kinds, `repinned`/`pruned_events` counts in doctor text and JSON, the updated and added `ref_maintenance` tests, the three already-updated reconciliation context files; plus `UnrecoverableCursor` display text that tells the operator to let one agent tool run and rerun `sce doctor --fix`. Out — `AGENTS.md`, glossary and heuristic documentation (T02), any cursor-recovery transition, plain-doctor detection. + - Dependencies: none + - Done when: AC1, AC2, AC3 and AC4 hold and the change lands as one commit. + - Verify: `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml -- ref_reconciliation reconciliation_fix json_ref_reconciliation json_completed_state outcomes_map`; `nix flake check`. + - Completed: 2026-10-16 + - Files changed: `cli/src/services/mutation_trace/runtime/ref_reconciliation.rs`, `cli/src/services/mutation_trace/runtime/ref_maintenance/tests/fail_closed.rs`, `cli/src/services/doctor/mod.rs`, `context/plans/missing-history-recovery.md` + - Result: The missing-pin repair had already landed in `8bcec9a1`; this task appended operator guidance to the `UnrecoverableCursor` message ("Let one agent tool run in this checkout so it records a present cursor, then rerun `sce doctor --fix`."), asserted it in `ref_reconciliation_absent_cursor_tree_fails_closed_without_pruning`, and replaced two `push_str(&format!(..))` calls in `reconciliation_completed_text` with `write!` to satisfy clippy `format_push_string`, which failed `nix flake check`. + - Verify outcomes: targeted test filter set passed (53 passed, 0 failed); AC1 and AC2 single-test runs each ran exactly one test and passed; AC3 `rg` returned no matches; AC4 `rg -n "checkout_id|clone_id" cli/src` returns only the pre-existing `ReconcileError::CheckoutIdentity(_) => "checkout_identity"` failure-kind mapping (`ref_maintenance.rs:206`), no migration file was added, and `resolve_worktree_id` still returns `main` (`git_snapshot.rs:686`); `nix flake check` passed after the clippy fix. + - Deviations: The plan assumed the repair was uncommitted; it was committed in `8bcec9a1`, so T01 lands as a follow-up commit containing only the guidance text, test assertion, and clippy fix. AC4's `rg` check matches pre-existing `checkout_identity`, which is not per-clone identity; no new identity was introduced. + - Context impact: localized — `UnrecoverableCursor` operator guidance in `context/cli/mutation-trace-ref-reconciliation.md`; no architecture, contract, or terminology change. + - Context synchronization: synced + +- [x] T02: `Document the missing-history heuristic and its limits` (status:done) + - Task ID: T02 + - Scope: In — describe the three cases, invariants I1–I5, and the heuristic limitation (detects missing history, not clone identity; existing trees prove nothing; an absent cursor clears after one hook boundary) in `context/cli/mutation-trace-ref-reconciliation.md`; add a `missing durable history` glossary entry; fix the drifted present-tense `checkout identity` glossary entries (lines describing `sce setup` creating `/sce/checkout-id`) to the removed framing used by `context/cli/checkout-identity.md`; update `context/context-map.md` annotations. Out — code changes. + - Dependencies: T01 + - Done when: AC5 holds. + - Verify: `rg -in "checkout identity" context/glossary.md` shows only removed framing; manual read of the reconciliation doc's case list. + - Completed: 2026-10-16 + - Files changed: `context/cli/mutation-trace-ref-reconciliation.md`, `context/glossary.md`, `context/context-map.md`, `context/plans/missing-history-recovery.md` + - Result: Added a `Missing durable history (heuristic)` subsection to the reconciliation doc covering the three cases, invariants I1–I5, the hook-path/plain-doctor boundaries, and the limitation that detection finds missing history, not clone identity, and that an existing tree is not evidence of an unchanged clone; reworded repair step 2 so an absent cursor is transient (one agent tool run, then rerun `sce doctor --fix`) instead of "does not belong to this checkout". Rewrote the `checkout identity` glossary entry to removed framing, added a `missing durable history` entry, and removed present-tense checkout-identity claims from the `repository identity`, `checkout registry`, `setup local bootstrap`, `ServiceLifecycle`, and `sce doctor` glossary entries. Extended the reconciliation annotation in `context/context-map.md`. + - Verify outcomes: `rg -in "checkout identity" context/glossary.md` matches only removed framing (lines 39, 41, 47); AC5 `rg -in "creates/reuses checkout identity|Stable UUIDv7 identifier assigned" context/glossary.md` returned no matches (exit 1); `context/cli/mutation-trace-ref-reconciliation.md:174` states "An existing tree is **not evidence of an unchanged clone**"; manual read confirmed the three-case list and I1–I5. + - Deviations: Also corrected the drifted checkout-identity claims in the `repository identity`, `setup local bootstrap`, `ServiceLifecycle`, and `sce doctor` glossary entries, which the AC5 check (`creates/reuses checkout identity` at the `setup local bootstrap` entry) required; moved the existing `TreeId` paragraph above the new subsection so it stays with the repair steps. + - Context impact: localized — reconciliation domain doc, glossary terminology (`missing durable history`, removed `checkout identity`), and context-map annotation; no code, architecture, or contract change. + - Context synchronization: synced + +## Open questions + +None. The value questions on cursor recovery and plain-doctor detection were resolved by dropping both. + +## Validation Report + +**Status:** validated +**Date:** 2026-10-16 + +### Commands run + +- `nix flake check` -> exit 0 (all checks passed) +- `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml ref_reconciliation_prunes_events_whose_trees_are_absent_and_repins_the_rest -- --list` -> exit 0 (1 test listed) +- `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml ref_reconciliation_prunes_events_whose_trees_are_absent_and_repins_the_rest` -> exit 0 (1 passed, 608 filtered out) +- `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml ref_reconciliation_absent_cursor_tree_fails_closed_without_pruning -- --list` -> exit 0 (1 test listed) +- `nix develop -c ./scripts/run-cli-cargo.sh test --manifest-path cli/Cargo.toml ref_reconciliation_absent_cursor_tree_fails_closed_without_pruning` -> exit 0 (1 passed, 608 filtered out) +- `rg -n "prune_mutation_events|repair_missing_pins" cli/src/services/hooks cli/src/services/mutation_trace/runtime/coordinator.rs` -> exit 1 (no matches) +- `ls cli/migrations/agent-trace-repository/` and `git status --short cli/migrations` -> exit 0 (001–007 only; no migration added or changed) +- `rg -n "checkout_id|clone_id" cli/src` -> exit 1 (no matches) +- `rg -in "creates/reuses checkout identity|Stable UUIDv7 identifier assigned" context/glossary.md` -> exit 1 (no matches) + +### Success-criteria verification + +- [x] AC1: `doctor --fix` prunes exactly events with absent trees, keeps the rest, re-pins, completes -> exact-name test listed once and passed +- [x] AC2: absent cursor fails closed with `unrecoverable_cursor` and operator guidance -> exact-name test listed once and passed; `UnrecoverableCursor` display text in `ref_reconciliation.rs` contains ``rerun `sce doctor --fix` `` +- [x] AC3: no hook boundary prunes or repairs -> `rg` returned no matches +- [x] AC4: no per-clone identity introduced -> `rg -n "checkout_id|clone_id" cli/src` returned no matches; migrations unchanged; `resolve_worktree_id` returns `main` for the primary worktree (`git_snapshot.rs:685-686`) +- [x] AC5: durable context describes the cases and the heuristic limit -> glossary `rg` returned no matches; `context/cli/mutation-trace-ref-reconciliation.md:141` states an existing tree is "not evidence of an unchanged clone" + +### Failed checks and follow-ups + +- None. + +### Residual risks + +- The reconciliation failure kind `checkout_identity` was renamed to `worktree_identity` (with `ReconcileError`/`ProtectedWorktreeError::CheckoutIdentity` -> `WorktreeIdentity`) after T01/T02 to clear AC4; consumers of the old failure-kind string must update. +- The `UnrecoverableCursor` message still says the cursor's durable history "does not belong to this checkout", which reads as a clone-identity claim the documented heuristic disclaims. diff --git a/context/sce/agent-trace-hooks-command-routing.md b/context/sce/agent-trace-hooks-command-routing.md index fa75afb4a..ab1e9ed97 100644 --- a/context/sce/agent-trace-hooks-command-routing.md +++ b/context/sce/agent-trace-hooks-command-routing.md @@ -49,7 +49,7 @@ Interactive `sce setup` persists the independent confirmations at `agent_trace.a - The preflight is invoked only when the attribution gate passes; when the gate does not pass, no DB read or staged-diff capture occurs. - `pre-commit` is a deterministic no-op entrypoint. - **`post-commit` is an active intersection entrypoint** (see [agent-trace-db.md](agent-trace-db.md)): - - Agent Trace DB access resolves repository-scoped storage from `agent_trace.repository_id` or the configured Git remote, creates/reuses checkout ID for diagnostics, then opens `/sce/repos//agent-trace.db` through the repository storage resolver. + - Agent Trace DB access resolves repository-scoped storage from `agent_trace.repository_id` or the configured Git remote, then opens `/sce/repos//agent-trace.db` through the repository storage resolver. - The resolver tries a no-migration open/readiness check plus repository metadata validation first and falls back to migration-running initialization when the repository DB is absent or schema metadata is incomplete. - Captures the current commit's patch from git using `capture_post_commit_patch_from_git()`. - Queries recent `diff_traces` patches from the past 7 days via `RepositoryAgentTraceDb::recent_diff_trace_patches()`. @@ -64,7 +64,7 @@ Interactive `sce setup` persists the independent confirmations at `agent_trace.a - `run_post_commit_agent_trace_flow_with(...)` prints the received remote URL to stderr as `post-commit remote_url=` before building/validating/persisting the Agent Trace payload. - At the current runtime boundary, parsed optional `vcs_type` is forwarded into the Agent Trace builder; when absent, top-level `vcs` metadata is omitted. - The run-flow path maps commit-time metadata to RFC3339 and calls `agent_trace::build_agent_trace_from_evidence(...)` with separated evidence: `direct_patch` is the combined recent patch (intersected against the post-commit patch inside the builder, unchanged), and `mutation_ai_patch` is the bounded mutation-history AI coverage for the committed lines direct evidence did not cover. Direct provenance (`model_id`, related sessions, top-level `tool`) still derives from direct evidence only; mutation-only coverage widens `ai` / `mixed` classification without provenance. See [agent-trace-minimal-generator.md](agent-trace-minimal-generator.md). - - The mutation-AI patch comes from `mutation_trace::runtime::resolve_post_commit_mutation_ai_patch(...)`, a read-only entrypoint that resolves the invoking worktree's *existing* checkout identity (`checkout::resolve_git_dir` + `checkout::read_checkout_id`), reads `HEAD^{tree}`, and — under the worktree lock — captures a commit attribution cut (`latest_mutation_event_revision`) so events produced after the commit cannot participate. It returns an empty patch — direct-only fallback — when identity, mutation history, the `HEAD` tree, or the cut is unavailable. It creates no identity and performs no mutation-cursor write. The `post_commit_patch_intersections` write stays direct-only (`patch::intersect_patches` of the combined recent patch against the post-commit patch); mutation evidence never reaches `diff_traces` or that intersection row. The causal-lineage replay mechanics live in [../cli/mutation-trace-agent-attribution.md](../cli/mutation-trace-agent-attribution.md). + - The mutation-AI patch comes from `mutation_trace::runtime::resolve_post_commit_mutation_ai_patch(...)`, a read-only entrypoint that resolves the invoking worktree's Git dir and topology-derived worktree ID (`resolve_git_dir` + `resolve_worktree_id`), reads `HEAD^{tree}`, and — under the worktree lock — captures a commit attribution cut (`latest_mutation_event_revision`) so events produced after the commit cannot participate. It returns an empty patch — direct-only fallback — when identity, mutation history, the `HEAD` tree, or the cut is unavailable. It creates no identity and performs no mutation-cursor write. The `post_commit_patch_intersections` write stays direct-only (`patch::intersect_patches` of the combined recent patch against the post-commit patch); mutation evidence never reaches `diff_traces` or that intersection row. The causal-lineage replay mechanics live in [../cli/mutation-trace-agent-attribution.md](../cli/mutation-trace-agent-attribution.md). - The same run-flow call now also forwards optional `tool_name` / `tool_version` from `PostCommitIntersectionFlowResult` into `AgentTraceMetadataInput`, so built post-commit payloads preserve tool metadata derived from recent parsed diff-trace rows. - The built Agent Trace payload includes top-level `metadata.sce.version` from the compiled `sce` CLI package version, always-emitted `metadata.sce.line_changes` touched-line attribution counts, and range-level `content_hash` values computed from touched post-commit hunk content before conversion to JSON. See [agent-trace-minimal-generator.md](agent-trace-minimal-generator.md) for the full payload contract. - The built Agent Trace payload is converted to JSON `Value` and validated via `agent_trace::validate_agent_trace_value(...)` before persistence.