You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(server): report dangling provider references instead of failing startup #4657
I run multi-replica OpenShell gateways with PostgreSQL. A sandbox whose spec references a provider that was later deleted shouldn't stop every gateway restart, on any release, until someone edits the database by hand.
Problem Statement
Before #4147, DeleteProvider took no mutation guard, so a delete racing an attach could leave a sandbox whose spec names a provider that no longer exists. #4147 stops new dangling references but doesn't repair the ones a database already has.
When no global policy is set, validate_provider_composition_startup_preflight (crates/openshell-server/src/grpc/policy.rs) runs before any listener binds (crates/openshell-server/src/lib.rs) and any error is fatal. It pages every sandbox in every workspace through validate_provider_composition_for_existing_sandboxes and builds each sandbox's provider layers. A missing provider fails the record lookup with FailedPrecondition ("provider 'X' not found"); the ? ends the loop, and startup fails.
Impact / Why This Matters
Every gateway restart against that database fails, so the fleet crashloops and the upgrade that would prevent new dangling references can't roll out. The affected sandbox already fails closed on its own: create fails fast on missing providers, and a supervisor can't sync an admitted configuration for it, so the workload stays unstarted.
Proposed Design
The preflight detects a missing provider from the lookup result (not by matching the error string), logs one warning per affected sandbox naming the workspace, sandbox, and providers, and continues validating the remaining sandboxes. Store and catalog errors stay fatal, and a sandbox whose composed policy would be invalid still blocks startup (the #2962 guard).
Acceptance Criteria
A database holding a dangling provider reference no longer stops the gateway from starting.
Alternatives Considered
A repair step that removes the missing references at startup: it needs a mutation guard and a CAS per sandbox, and it changes user data at startup, while the affected sandbox already fails closed on its own.
Activity