Skip to content

feat(azure): complete Phase 3 pilot with staged blocks, conditional uploads, and metadata - #192

Merged
skyoo2003 merged 3 commits into
mainfrom
feat/phase-3-azure-pilot-completion
Oct 11, 2026
Merged

skyoo2003 merged 3 commits into
mainfrom
feat/phase-3-azure-pilot-completion

Conversation

@skyoo2003

Copy link
Copy Markdown
Owner

Summary

Completes the Phase 3 Azure Blob Storage pilot and multi-CSP v2.0 groundwork by adding staged block uploads, conditional requests, and custom metadata handling, accompanied by deterministic codegen and expanded Python SDK compatibility tests.

Related Issue

Fixes #

Changes

  • OpenAPI Model & Deterministic Codegen:
    • Expanded api/models/azure-blob-storage.json with PutBlock, PutBlockList, and SetBlobMetadata operations and associated schemas (BlobAlreadyExists, ConditionNotMet, input/output types).
    • Fixed map iteration non-determinism in internal/codegen/openapi_parser.go by lexicographically sorting schema and property keys.
    • Added parser determinism and route matching tests in internal/codegen/openapi_parser_test.go.
  • Azure Blob Storage Provider & Engine:
    • Implemented conditional request handling in internal/services/blob/store.go and provider.go (If-None-Match: * returning 409 BlobAlreadyExists, If-Match returning 412 ConditionNotMet).
    • Added containment-safe block staging (StageBlock) and atomic assembly (CommitBlockList) with path traversal guards.
    • Implemented user metadata storage (SetBlobMetadata) and header response preservation in internal/gateway/router.go (preserving lowercase x-ms-meta-* headers for Azure SDK deserialization).
  • SDK Compatibility Tests:
    • Expanded test/compatibility/test_azure_blob.py with tests for conditional upload, staged block upload, custom metadata, and byte-range downloads (6 of 6 tests passing).
  • Documentation:
    • Updated docs/roadmap.md marking Phase 3 complete with 2.x ← current versioning.
    • Updated docs/services/azure.md with new operations and Python SDK usage examples.
    • Synchronized published figures in docs/coverage.md.

Test Plan

  • go test ./...: All unit and integration tests passing.
  • make codegen: Verified 100% deterministic code generation with zero diff churn.
  • pytest test/compatibility/test_azure_blob.py: All 6 Azure Blob Storage SDK tests passing.
  • python3 ./scripts/check-site-links.py: All 1,202 internal documentation links resolve.
  • pre-commit run --all-files: All linters, formatting, and safety checks passing.

Checklist

  • Self-reviewed the code
  • Added/updated tests
  • Lint/format passes (golangci-lint run)
  • Updated documentation (if applicable)
  • Added a Changie changelog fragment for user-facing changes (changie new, one sentence and two at most, see RELEASE.md) — or N/A (docs/tests/chore only)

…ploads, and metadata

- Expand OpenAPI model with PutBlock, PutBlockList, and SetBlobMetadata operations
- Fix codegen non-determinism by sorting schemas and properties in openapi_parser
- Support conditional requests (If-None-Match, If-Match) in FileStore and Provider
- Implement chunked block staging and atomic block list commitment with path traversal guards
- Support user-defined blob metadata and preserve lowercase x-ms-meta-* response headers
- Expand azure-storage-blob compatibility tests covering new operations
- Sync docs/roadmap.md, docs/services/azure.md, and docs/coverage.md
@github-actions github-actions Bot added documentation Improvements or additions to documentation tests Test code and test infrastructure codegen Smithy codegen and generated code services AWS service implementations labels Oct 11, 2026
@skyoo2003
skyoo2003 merged commit f7b1338 into main Oct 11, 2026
11 checks passed
@skyoo2003
skyoo2003 deleted the feat/phase-3-azure-pilot-completion branch October 11, 2026 09:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codegen Smithy codegen and generated code documentation Improvements or additions to documentation services AWS service implementations tests Test code and test infrastructure

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant