Skip to content

feat: implement Phase 4 multi-CSP breadth expansion (GCS, Azure, Federated Identity) - #193

Merged
skyoo2003 merged 3 commits into
mainfrom
feat/phase-4-breadth-expansion
Oct 11, 2026
Merged

skyoo2003 merged 3 commits into
mainfrom
feat/phase-4-breadth-expansion

Conversation

@skyoo2003

Copy link
Copy Markdown
Owner

Summary

This PR implements Phase 4 ("Breadth expansion") of the DevCloud roadmap:

  1. Google Cloud Storage (GCS) Pilot:

    • GCP Bearer/OAuth token authentication support (internal/auth/gcp_auth.go)
    • OpenAPI model definition (api/models/gcp-storage.json) with path routing for bucket/object operations ({object+})
    • SQLite-backed store with Castagnoli CRC32c and MD5 checksum computation
    • Compatibility integration suite (test/compatibility/test_gcp_storage.py)
  2. Azure Storage Expansion (Queue & Table Storage):

    • Azure Queue Storage (azurequeue): XML response rendering, PutMessage, PeekMessages, GetMessages, DeleteMessage, ClearMessages, GetQueueMetadata
    • Azure Table Storage (azuretable): OData JSON (application/json;odata=minimalmetadata), table CRUD, and entity CRUD operations
    • Compatibility suites: test/compatibility/test_azure_queue.py and test/compatibility/test_azure_table.py
  3. Azure Cosmos DB NoSQL REST API:

    • SQLite-backed DocumentDB REST API (cosmosdb): database CRUD, collection CRUD, and document CRUD with partition key routing
    • Compatibility suite: test/compatibility/test_azure_cosmos.py
  4. Federated Identity Playground:

    • Local OIDC issuer (internal/services/iam/oidc_issuer.go) with /.well-known/openid-configuration, /jwks.json, and /token minting signed RS256 JWTs
    • Cross-CSP exchange endpoints:
      • Azure AD token exchange (/common/oauth2/v2.0/token)
      • Google Cloud STS token exchange (/v1/sts/token)
    • AWS STS federation (sts:AssumeRoleWithWebIdentity): validated against minting issuer and claims extraction
    • Compatibility suite: test/compatibility/test_federated_identity.py
  5. Single-Port Sniffing & Routing Updates:

    • Refined single-port 4747 protocol detection in internal/gateway/protocol.go to seamlessly disambiguate blob, azurequeue, azuretable, cosmosdb, and identity without collisions
  6. Documentation & Manifests Synchronization:

    • Updated docs/roadmap.md, docs/coverage.md, docs/architecture.md, README.md, and docs/fidelity-manifest.md
    • Added docs/services/gcp.md and docs/services/federated-identity.md

Verification

  • Go Unit Tests:
    • go test ./internal/auth ./internal/gateway ./internal/config ./internal/services/... (All passed)
  • Integration Tests:
    • pytest test/compatibility/test_{azure_blob,azure_queue,azure_table,azure_cosmos,gcp_storage,federated_identity}.py (14/14 passed)
  • Regression Tests:
    • AWS core services: pytest test/compatibility/test_{iam_sts,s3,sqs,dynamodb}.py (86/86 passed)
  • Fidelity & Manifest Gating:
    • go test ./cmd/devcloud (Passed with manifest consistency gates)

…rated Identity)

- GCS Pilot: added GCP auth, OpenAPI model, storage provider, Castagnoli CRC32c/MD5 checksums, and compatibility tests
- Azure Expansion: implemented Queue Storage and Table Storage providers with XML/OData JSON responses and SQLite stores
- Azure Cosmos DB: added SQLite-backed DocumentDB REST API for databases, collections, and documents with partition key routing
- Federated Identity: added local OIDC issuer (RS256 JWT, discovery, JWKS), Azure AD/GCP STS token exchanges, and STS AssumeRoleWithWebIdentity integration
- Protocol Router: updated single-port (4747) sniffing logic to seamlessly distinguish Azure Blob, Queue, Table, Cosmos DB, and Identity endpoints
- Documentation & Fidelity: regenerated manifests, updated roadmap, coverage, and service guides
@github-actions github-actions Bot added documentation Improvements or additions to documentation tests Test code and test infrastructure codegen Smithy codegen and generated code services AWS service implementations labels Oct 11, 2026
Comment thread internal/services/gcs/store.go Fixed
Comment thread internal/services/gcs/store.go Fixed
…e test failures

- golangci-lint: wrap rows.Close() in defer func with blank assignment in azuretable and cosmosdb stores
- golangci-lint: omit redundant type int from expSec in identity provider
- packaging: move identity service from internal/services/iam to dedicated internal/services/identity package
- imports: update cmd/devcloud/imports.go via scripts/generate-imports.sh to include identity
- exclusions: add Phase 4 non-AWS services to exclusions.json under noBoto3Client and update coverage.md figure to 427
@skyoo2003
skyoo2003 merged commit 27b3715 into main Oct 11, 2026
11 checks passed
@skyoo2003
skyoo2003 deleted the feat/phase-4-breadth-expansion branch October 11, 2026 22:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codegen Smithy codegen and generated code documentation Improvements or additions to documentation services AWS service implementations tests Test code and test infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants