Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
442 changes: 441 additions & 1 deletion api/models/azure-blob-storage.json

Large diffs are not rendered by default.

5 changes: 5 additions & 0 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,3 +193,8 @@ touched an *addition* rather than an edit:
The plugin system, protocol detector and storage abstractions stay CSP-agnostic
by convention rather than enforcement — `ProtocolType` is an open string type,
not an enum, for exactly this reason.

Phase 3 established the first non-AWS pilot (**Azure Blob Storage**) exercising all
four seams: OpenAPI models in `api/models/` parsed via `OpenAPISource`, namespaced
configuration under `providers.azure.services.blob`, `ProviderScoped` plugin registration,
and `AzureSharedKey` authentication.
6 changes: 3 additions & 3 deletions docs/coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,10 @@ denominators, because [routing and depth are two targets](#the-target):

| Tier | Serving target | All registered |
|---|---|---|
| `hand-verified` | 4,611 | 4,642 |
| `auto-crud` | 7,530 | 14,083 |
| `hand-verified` | 4,620 | 4,651 |
| `auto-crud` | 7,529 | 14,082 |
| `unimplemented` | 267 | 477 |
| **total known** | **12,408** | **19,202** |
| **total known** | **12,416** | **19,210** |
| **hand-verified share** | **37.2%** | **24.2%** |

The **serving target** column is the depth promise: every registered service
Expand Down
71 changes: 67 additions & 4 deletions docs/services/azure.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,79 @@
# Azure Services

DevCloud is expanding its local development capabilities to include Microsoft Azure services, starting with Azure Blob Storage. This represents the beginning of our multi-CSP architectural vision.
DevCloud expands its local development runtime to include Microsoft Azure services, starting with Azure Blob Storage as the Phase 3 multi-CSP pilot.

## Supported Services

| Service | Supported | Local limits |
|---|---|---|
| Blob Storage | Initializing | Mock engine returning generic responses; strict semantic verification in progress. |
| Blob Storage (`blob`) | Supported (Phase 3 pilot) | Local filesystem-backed FileStore under `data/blob`. Supports container CRUD, BlockBlob CRUD, listing/enumeration, and byte-range downloads. |

## Credentials
DevCloud accepts Azure Shared Key credentials without validating signatures. For local development, use any dummy account key (or the standard local development key):
## Operations

| Area | Operation | Method & URI | Notes |
|---|---|---|---|
| Containers | `CreateContainer` | `PUT /{container}?restype=container` | Returns `201 Created` or `409 ContainerAlreadyExists` |
| Containers | `DeleteContainer` | `DELETE /{container}?restype=container` | Returns `202 Accepted` or `404 ContainerNotFound` |
| Containers | `GetContainerProperties` | `GET /{container}?restype=container` | Returns lease state and metadata headers |
| Containers | `ListContainers` | `GET /?comp=list` | Returns XML `EnumerationResults` listing all containers |
| Blobs | `PutBlob` | `PUT /{container}/{blob}` | Creates or replaces a BlockBlob; stores payload and computes MD5 |
| Blobs | `GetBlob` | `GET /{container}/{blob}` | Downloads blob content; supports `Range` and `x-ms-range` requests |
| Blobs | `GetBlobProperties` | `HEAD /{container}/{blob}` | Returns headers (`x-ms-blob-type: BlockBlob`, size, ETag, Content-MD5) |
| Blobs | `DeleteBlob` | `DELETE /{container}/{blob}` | Permanently removes the blob |
| Blobs | `ListBlobs` | `GET /{container}?restype=container&comp=list` | Returns XML `EnumerationResults` listing blobs in the container |

## Configuration

In `devcloud.yaml`, Azure services are configured under the namespaced `providers.azure` block:

```yaml
server:
port: 4747

providers:
azure:
services:
blob:
enabled: true
data_dir: ./data/blob
```

## Credentials & Connection Strings

DevCloud accepts Azure Shared Key, SharedKeyLite, and Azurite emulator credentials without signature validation:

```bash
export AZURE_STORAGE_CONNECTION_STRING="DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:4747/devstoreaccount1;"
```

## Python SDK Quickstart

Using `azure-storage-blob`:

```python
from azure.storage.blob import BlobServiceClient

connection_string = (
"DefaultEndpointsProtocol=http;"
"AccountName=devstoreaccount1;"
"AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;"
"BlobEndpoint=http://127.0.0.1:4747/devstoreaccount1;"
)

client = BlobServiceClient.from_connection_string(connection_string)

# 1. Create a container
container_client = client.create_container("my-container")

# 2. Upload a BlockBlob
blob_client = container_client.get_blob_client("notes.txt")
blob_client.upload_blob(b"Hello from local DevCloud Azure Blob Storage!")

# 3. Download the blob
content = blob_client.download_blob().readall()
print(content.decode("utf-8"))

# 4. List blobs
for blob in container_client.list_blobs():
print(f"Blob: {blob.name}, Size: {blob.size}")
```
55 changes: 45 additions & 10 deletions internal/auth/azure_shared_key.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,18 +13,53 @@ func (AzureSharedKey) Provider() string { return "azure" }

func (s AzureSharedKey) Authenticate(r *http.Request) (Identity, bool) {
authz := r.Header.Get("Authorization")
if !strings.HasPrefix(authz, "SharedKey ") {
return Identity{}, false
if strings.HasPrefix(authz, "SharedKey ") {
parts := strings.SplitN(authz[10:], ":", 2)
if len(parts) == 2 {
return Identity{
Provider: s.Provider(),
AccessKeyID: parts[0],
Service: "blob",
}, true
}
}
if strings.HasPrefix(authz, "SharedKeyLite ") {
parts := strings.SplitN(authz[14:], ":", 2)
if len(parts) == 2 {
return Identity{
Provider: s.Provider(),
AccessKeyID: parts[0],
Service: "blob",
}, true
}
}

q := r.URL.Query()
if q.Get("sig") != "" && (q.Get("se") != "" || q.Get("sp") != "" || q.Get("sv") != "") {
account := "devstoreaccount1"
trimmed := strings.Trim(r.URL.Path, "/")
if parts := strings.Split(trimmed, "/"); len(parts) > 0 && parts[0] != "" {
account = parts[0]
}
return Identity{
Provider: s.Provider(),
AccessKeyID: account,
Service: "blob",
}, true
}

parts := strings.SplitN(authz[10:], ":", 2)
if len(parts) != 2 {
return Identity{}, false
if r.Header.Get("x-ms-version") != "" || strings.HasPrefix(r.URL.Path, "/devstoreaccount1") {
account := "devstoreaccount1"
trimmed := strings.Trim(r.URL.Path, "/")
if parts := strings.Split(trimmed, "/"); len(parts) > 0 && parts[0] != "" {
account = parts[0]
}
return Identity{
Provider: s.Provider(),
AccessKeyID: account,
Service: "blob",
}, true
}

return Identity{
Provider: s.Provider(),
AccessKeyID: parts[0],
Service: "blob",
}, true
return Identity{}, false
}
57 changes: 57 additions & 0 deletions internal/auth/azure_shared_key_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
// SPDX-License-Identifier: Apache-2.0

package auth

import (
"net/http"
"net/http/httptest"
"testing"

"github.com/stretchr/testify/assert"
)

func TestAzureSharedKey_Authenticate(t *testing.T) {
adapter := AzureSharedKey{}

t.Run("SharedKey header", func(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/mycontainer", nil)
req.Header.Set("Authorization", "SharedKey devstoreaccount1:abcdef==")
id, ok := adapter.Authenticate(req)
assert.True(t, ok)
assert.Equal(t, "azure", id.Provider)
assert.Equal(t, "devstoreaccount1", id.AccessKeyID)
assert.Equal(t, "blob", id.Service)
})

t.Run("SharedKeyLite header", func(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/mycontainer", nil)
req.Header.Set("Authorization", "SharedKeyLite myaccount:secret123==")
id, ok := adapter.Authenticate(req)
assert.True(t, ok)
assert.Equal(t, "azure", id.Provider)
assert.Equal(t, "myaccount", id.AccessKeyID)
})

t.Run("SAS query parameters", func(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/myaccount/mycontainer?sig=sig123&se=2026-10-10", nil)
id, ok := adapter.Authenticate(req)
assert.True(t, ok)
assert.Equal(t, "azure", id.Provider)
assert.Equal(t, "myaccount", id.AccessKeyID)
})

t.Run("x-ms-version header", func(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/devstoreaccount1/mycontainer", nil)
req.Header.Set("x-ms-version", "2023-11-03")
id, ok := adapter.Authenticate(req)
assert.True(t, ok)
assert.Equal(t, "azure", id.Provider)
assert.Equal(t, "devstoreaccount1", id.AccessKeyID)
})

t.Run("unrecognized request", func(t *testing.T) {
req := httptest.NewRequest(http.MethodGet, "/bucket/key", nil)
_, ok := adapter.Authenticate(req)
assert.False(t, ok)
})
}
Loading
Loading