Skip to content

refactor(providers)!: make provider behavior explicit in imported profiles #3442

Open
#3775
Open
refactor(providers)!: make provider behavior explicit in imported profiles#3442
#3775
Labels
state:triage-neededOpened without agent diagnostics and needs triage

Description

@feloy
feloy
opened on Sep 18, 2026

Sub-issue of #3171 (provider boundary in step 5). Follows #3299 / PR #3383, which made provider profiles import-only. Coordinated pre-0.1.0 breaking change under #2565.

User Story

As an operator, I want an imported provider profile to declare every behavior it activates, so I can inspect, fork, and rename the profile without silently changing the sandbox environment.

Problem Statement

#3299 removed the compiled provider profile catalog, but two compiled adapters remain:

  • google-cloud projects provider config into GCP SDK variables and sets GCE_METADATA_HOST.
  • google-vertex-ai projects project and region config into GCP and Vertex variables and sets a GOOSE_PROVIDER default.

ProviderRegistry::inject_env_for_profile_id selects these adapters by the resolved profile ID. Importing the same profile as google-cloud and acme-gcp therefore produces different environments even though the imported definitions are identical.

Other provider-specific behavior is also compiled rather than declared: Vertex discovery scans a fixed list of config variables, and child-environment preparation contains GCP-specific metadata and non-secret configuration handling. ProviderProfile can declare credential environment variables, but not non-secret config projection, fixed non-secret values, discovery config keys, or a required platform adapter.

Impact / Why This Matters

An imported profile is not currently its complete definition. Operators must preserve canonical IDs and know release-specific implementation details, or provider creation succeeds while the workload later fails because expected SDK configuration is absent. This contradicts the import-only contract and makes profile forks unsafe.

Proposed Design

Inventory every compiled provider behavior reachable from an imported profile and give it one disposition:

  1. Express in the profile. Add bounded declarative fields for generic non-secret behavior such as config-to-environment projection, fixed values, and discovery config keys. Do not introduce templating or scripting.
  2. Declare a platform adapter. If behavior is genuinely platform-specific, the profile names the required adapter. Import or attachment fails with a bounded diagnostic when the adapter is unavailable.
  3. Remove it. Delete behavior or APIs with no remaining caller, including ProviderDiscoverySpec and discover_with_spec unless a supported use is identified.

Projection must preserve the existing rule that caller-supplied environment values win. Linting must reject collisions between non-secret projection and credential env_vars.

Update the Google Cloud and Vertex examples so their YAML describes their complete environment and discovery effects. A fork imported under a different ID must behave identically to the canonical example.

Credential refresh strategies are out of scope because profiles already declare them.

Acceptance Criteria

  • Every compiled provider behavior reachable from an imported profile is inventoried as profile-declared, named platform adapter, or removed.
  • No provider behavior activates from the profile ID alone.
  • The Google Cloud and Vertex examples declare their complete non-secret environment and discovery effects.
  • A profile forked under another ID produces the same sandbox environment and discovery behavior as the canonical profile.
  • Environment projection never overwrites an existing value.
  • Import or attachment fails with a bounded diagnostic when a declared platform adapter is unavailable.
  • Profile lint rejects unknown adapters and collisions between non-secret projection and credential environment variables.
  • ProviderDiscoverySpec and discover_with_spec are removed or have a documented supported caller.
  • ProviderProfile.source and resource_version documentation no longer refers to built-in profiles or builtin provenance.
  • Provider examples, architecture/user documentation, and 0.1.0 migration notes describe the resulting contract.
  • Tests cover profile/proto round trips, forked-ID equivalence, collision handling, non-overwrite semantics, and unavailable adapters.

Alternatives Considered

  • Delete both adapters: restores a clean boundary but breaks existing GCP and Vertex workloads at upgrade.
  • Document ID-keyed behavior only: leaves profile forks behaviorally different and the binary authoritative for part of the profile definition.
  • Add a general templating language: unnecessarily turns reviewable profile data into executable configuration; bounded projection covers the observed generic cases.
  • Restore aliases: cannot support an unbounded operator catalog and reintroduces hidden ID coupling.

Technical Notes

  • crates/openshell-providers/src/lib.rs: ProviderRegistry registers the two remaining adapters and selects them by exact profile ID.
  • crates/openshell-providers/src/discovery.rs: discover_from_profile special-cases google-vertex-ai config keys.
  • crates/openshell-core/src/provider_credentials.rs: child environment preparation contains GCP-specific metadata and non-secret resolution.
  • crates/openshell-server/src/grpc/provider.rs: provider environment assembly and key-collision validation are the runtime integration points.
  • proto/openshell.proto and crates/openshell-providers/src/profiles.rs: the public profile schema and YAML/protobuf conversion currently lack this declaration surface.

Checklist

  • Existing issues and architecture documentation reviewed
  • Design proposal; implementation planning follows human disposition

Activity

  1. github-actions
    added
    state:triage-neededOpened without agent diagnostics and needs triage
    on Sep 18, 2026
  2. feloy
    changed the title [-]refactor(providers)!: inventory type-specific provider adapters and classify built-in middleware[/-] [+]refactor(providers)!: make provider behavior explicit in imported profiles[/+] on Sep 23, 2026

  3. feloy commented on Sep 28, 2026

    @feloy
    feloy
    on Sep 28, 2026
    ContributorAuthor

    🏗️ build-plan

    Implementation Plan

    Issue type: refactor
    Complexity: High
    Confidence: High — the profile, gateway, and sandbox paths have been verified; history confirms PR #2942 removed the GCP metadata server.

    Summary

    Make imported profiles the authority for non-secret environment projection and local discovery. Declare the unavailable GCP metadata capability explicitly, remove exact-ID switches and unused APIs, and make a renamed profile behave like its source. Preserve caller environment values, exclude legacy Vertex service-account private keys without withholding the rest of the provider environment, and allow multiple providers to share identical non-secret defaults.

    Scope

    • Add bounded profile declarations for config-to-environment projection, fixed non-secret values, discovery config keys, and a required platform adapter. Keep profile revision encoding deterministic and regenerate the tracked Go protobuf bindings and domain converters.
    • Apply declarations in the gateway without profile-ID switches. Sandbox template and create-time environment values, including empty values, take precedence over non-secret profile defaults in both main and exec paths; provider credential placeholders remain provider-owned.
    • Reject credential-key collisions across attached providers. Accept shared non-secret destination keys only when the projected values are identical; reject differing values with the key and both provider names.
    • Reject new import, update, and lint requests that declare GOOGLE_SERVICE_ACCOUNT_KEY as a provider credential or as an environment.config or environment.fixed destination. Keep older stored profiles loadable while excluding that key from workload environment, credential bindings, and proxy substitution. Skip a stale stored private key before undeclared-credential readiness checks so the access token and non-secret SDK settings remain available after the declaration is removed. Continue to supply private key material through credential refresh configuration.
    • Declare gcp-metadata in the Google Cloud example and fail import/update/attachment while the adapter is unavailable. Vertex bearer-token authentication remains usable.
    • Remove the ID-selected Google Cloud and Vertex adapters, orphaned metadata helpers, unused discovery API, and CLI hints selected by profile ID. Update examples, architecture docs, published docs, migration notes, and the affected operator skill.

    Implementation Steps

    1. Complete: Add profile/protobuf declarations, bounded validation, YAML/protobuf round trips, deterministic maps, and SDK bindings.
    2. Complete: Replace ID-selected projection and discovery with profile-selected behavior; verify canonical and forked IDs and adapter errors.
    3. Complete: Filter non-secret profile defaults when callers supply the same sandbox environment key. Verify main and exec launch paths, including explicit empty values and credential placeholders.
    4. Complete: Reject differing cross-provider non-secret values in both attachment orders while accepting identical Vertex defaults and distinct credential keys. Skip stale private-key credentials before readiness checks, including after a profile removes the legacy declaration. Reject the legacy key in credential declarations and non-secret defaults, and suppress it from older stored profiles without invalidating the catalog.
    5. Complete: Document atomic directory imports and use selected compatible examples; update the cluster debug skill, provider guides, architecture overview, and upgrade guidance.
    6. Complete: Publish fix-3442/vertex at signed-off commit 5e32a9462 and open PR #3775 for review.

    Verification

    • mise run pre-commit passed after the final edits and in the amend hook.
    • mise run test passed after the Vertex fixes, including the gateway test-support run and the wider Rust, Python, TypeScript, and repository checks.
    • mise run ci passed on the amended commit before PR creation.
    • All 156 focused gateway provider tests passed. New regressions cover import rejection for both non-secret legacy-key defaults, retaining token and SDK values when an undeclared private key remains stored, and attaching two Vertex providers with identical defaults and distinct credential keys. The provider projection test confirms old stored defaults cannot emit the private-key variable.
    • The focused Podman provider-refresh lane and six CLI conformance scenarios passed for the profile projection and caller-precedence implementation before the later gateway-only safeguards. The later safeguards were covered by the gateway and provider tests above.

    Risks and Operational Notes

    • Commit c1f2e7189f837f98d00fb3821b6fb8d1253ceec3 (September 16, PR feat(isolation): implement the RFC 0012 sandbox architecture #2942) deleted the metadata HTTP server and startup integration. A batch import containing providers/google-cloud.yaml fails atomically until gcp-metadata is implemented; import selected compatible profiles instead.
    • Existing stored profiles that declare GOOGLE_SERVICE_ACCOUNT_KEY remain readable but cannot inject it. Removing the declaration while a provider record still holds the key leaves that key excluded without marking the whole snapshot withheld; the access token and SDK configuration remain available. Operators should migrate private key material to refresh configuration.
    • Two attached providers may share a non-secret destination when both projected values match. Different values and overlapping credential keys still fail validation; operators can use distinct destination names or attach one provider in those cases. A sandbox environment override does not bypass the provider collision check.

    Documentation Impact

    Updated architecture/google-vertex-ai-provider.md, docs/how-it-works/providers/profiles.mdx, docs/how-it-works/providers/google.mdx, docs/upgrade/0-1-0.mdx, providers/README.md, and skills/debug-openshell-cluster/SKILL.md. No gateway TOML or Helm field changes; no LSM or /proc behavior changes.

    Revision 5 — published PR #3775 and verified full local CI on the amended commit
    Revision 4 — preserve Vertex snapshots after legacy declaration removal, reject private-key defaults, and allow equal-value provider overlaps
    Revision 3 — implementation results, legacy private-key safety, deterministic collision rejection, and compatible example imports
    Revision 2 — verified metadata removal, deterministic profile serialization, and explicit refresh-material migration
    Revision 1 — initial plan

  4. johntmyers
    added 3 commits that reference this issue on Sep 29, 2026

    refactor(providers)!: declare imported profile behavior safely

    1ef3553

    refactor(providers)!: declare imported profile behavior safely

    c8d0d5a

    refactor(providers)!: declare imported profile behavior safely

    e3ba552
  5. feloy
    added 5 commits that reference this issue on Sep 30, 2026

    refactor(providers)!: declare imported profile behavior safely

    adb2273

    refactor(providers)!: declare imported profile behavior safely

    b0b15d6

    refactor(providers)!: declare imported profile behavior safely

    b69f39c

    refactor(providers)!: declare imported profile behavior safely

    dd89d33

    refactor(providers)!: declare imported profile behavior safely

    df15133
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    state:triage-neededOpened without agent diagnostics and needs triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions