From 6301f2ae062782422e122a7f9add9033e9639aee Mon Sep 17 00:00:00 2001 From: Sung-Kyu Yoo Date: Sat, 10 Oct 2026 08:56:48 +0900 Subject: [PATCH] docs: sync Hugo documentation with v1.3.0 implementation state This commit updates the Hugo documentation to accurately reflect the codebase state following the Phase 1A-1F integrations: - Add EventBridge and SNS to the Hugo sidebar menu - Update outdated operation counts (auto-crud: 14,082, hand-verified: 4,642) - Remove outdated rds-data exceptions and phantom IAM policy enforcement text - Update API operation support tables and limitations for S3 (43), DynamoDB (29), Lambda (35) - Standardize EventBridge and SNS pages with API tables and boto3/CLI code examples --- docs/README.md | 2 +- docs/compatibility-policy.md | 8 ++--- docs/coverage.md | 17 ++++++----- docs/crud-engine.md | 1 + docs/demand.md | 5 ++-- docs/faq.md | 4 +-- docs/fidelity-manifest.md | 4 +-- docs/services-matrix.md | 4 ++- docs/services/dynamodb.md | 13 +++----- docs/services/eventbridge.md | 58 ++++++++++++++++++++++++++++++++++++ docs/services/lambda.md | 7 +++-- docs/services/s3.md | 2 +- docs/services/sns.md | 56 +++++++++++++++++++++++++++++++++- hugo.toml | 10 +++++++ 14 files changed, 157 insertions(+), 34 deletions(-) diff --git a/docs/README.md b/docs/README.md index ddd5c01d..a18a5706 100644 --- a/docs/README.md +++ b/docs/README.md @@ -18,7 +18,7 @@ | [Fidelity Manifest](fidelity-manifest.md) | Per-operation tiers: how much to trust any given call | | [CRUD Engine](crud-engine.md) | How engine-served operations behave, and where they stop | | [Services Matrix](services-matrix.md) | Depth by group, cross-service integrations, protocols | -| [Demand](demand.md) | Which unregistered services have demonstrated demand, and how it was measured | +| [Demand](demand.md) | Which services have demonstrated demand, and how it was measured | ## Per-service references diff --git a/docs/compatibility-policy.md b/docs/compatibility-policy.md index 68fbbe58..52cc6adf 100644 --- a/docs/compatibility-policy.md +++ b/docs/compatibility-policy.md @@ -116,12 +116,12 @@ promise means adding or strengthening assertions, and such contributions are wel Depending on any of the following will break, and breaking it is **not** a major-version event. -- **`auto-crud` response content.** 10,871 operations are served by the +- **`auto-crud` response content.** 14,082 operations are served by the [generic CRUD engine](crud-engine.md) at fidelity that is deliberately *plausible, not faithful*: store-backed responses echoing your input plus synthesized ids and ARNs, with no validation, no cross-resource integrity, no pagination correctness and no business logic. Their shape and content may change in any release. Use them to wire an SDK up, nothing more. -- **Hand-verified operations with no compatibility test.** Of 4,528 hand-verified operations, +- **Hand-verified operations with no compatibility test.** Of 4,642 hand-verified operations, only what the suite covers is promised. The rest are best-effort. - **Data durability.** Stores are local development stores. Several are in-memory and per-process; on-disk layouts under `data_dir` may change format between releases without a @@ -129,8 +129,8 @@ Depending on any of the following will break, and breaking it is **not** a major - **`unimplemented` → served transitions.** An operation that returns an error today may start returning a response. This is additive, and ships in a minor release. - **Service coverage.** New services may be added in a minor release. The 431 services registered - today are a floor, not a ceiling — and not a promise of depth either: 5 of them serve no - operation and only decline cleanly. See [coverage.md](coverage.md). + today are a floor, not a ceiling — and all of them serve at least one operation. + See [coverage.md](coverage.md). - **Error codes, HTTP status and message wording.** What *is* guaranteed for an `unimplemented` operation is that it **fails** — an AWS-shaped error, never a fabricated success. *Which* error is not guaranteed: it comes from whichever provider handles the request, and `sqs` even differs diff --git a/docs/coverage.md b/docs/coverage.md index 785a7d9c..88ea0361 100644 --- a/docs/coverage.md +++ b/docs/coverage.md @@ -69,9 +69,11 @@ the parser does not read. **The operation is not CRUD-shaped.** `GetThing`, `ListThings` and `CreateThing` map onto a generic store. `ExecuteStatement`, `InvokeEndpoint` and `QueryForecast` do not, and the engine refuses them rather than inventing an -answer. This applies to four services: `forecastquery`, the two SageMaker Runtime -variants `sagemaker-runtime` and `sagemakerruntimehttp2`, and `rds-data`. No -protocol change reaches them. +answer. This applies to three services: `forecastquery` and the two SageMaker +Runtime variants `sagemaker-runtime` and `sagemakerruntimehttp2`. No protocol +change reaches them; instead, each has a hand-written provider that serves at +least one operation natively. `rds-data` is served by a SQLite-backed query +engine added in Phase 1E. EC2 (`ec2Query`) and Partner Central Revenue Measurement (`rpcv2Cbor`) originally had protocols the parser did not read; both protocols and native CBOR serialization are now @@ -195,11 +197,10 @@ evidence in [demand.md](demand.md); re-derive with The rule kept the 100% depth target only if ≥60% of `M` had support ≥2, and narrowed to a demand set if ≥100 did. 57 cleared neither bar, so the pre-registered consequence applied: **the 100% depth claim is dropped and the -published depth target becomes the demand set.** All 57 are registered — 56 serve -at least one operation, and `rds-data` is the exception named above. It is -supported by all three projects, the strongest signal in the set, and still -cannot be served generically. The engine does not reach every service it routes, -and saying so is cheaper than a fabricated success. +published depth target becomes the demand set.** All 57 are registered and serve +at least one operation — `rds-data`, the strongest-signal service in the set, is +now served by a SQLite-backed query engine (Phase 1E) rather than the generic +CRUD engine. Four fifths of the AWS surface is surface that three projects with far more history and staffing have collectively declined to build. That is what a long diff --git a/docs/crud-engine.md b/docs/crud-engine.md index ede88d3b..7c126043 100644 --- a/docs/crud-engine.md +++ b/docs/crud-engine.md @@ -37,6 +37,7 @@ The engine has to know which operation a request is for before it can classify i | `rest-xml` | the same — every `restXml` operation binds to a method and URI | yes | | `query` | the `Action` field of the form body | yes | | `ec2-query` | the `Action` field of the form body or query parameter | yes | +| `rpcv2-cbor` | the path — RPC-v2 routes by service and operation name segments | yes | Every protocol DevCloud registers is readable, so a service that serves nothing does so because none of its operations is CRUD-shaped, not because of how it diff --git a/docs/demand.md b/docs/demand.md index e8e139d4..7775fa36 100644 --- a/docs/demand.md +++ b/docs/demand.md @@ -1,4 +1,4 @@ -# Demand for unregistered services +# Demand study The evidence behind DevCloud's [coverage target](coverage.md#the-target). Sampled 2026-09-05; re-derive with `python3 scripts/demand_rank.py`. @@ -46,8 +46,7 @@ at any point would have stopped on the best surface available. **All 57 services with support ≥ 2 are registered**, down to and including `workspaces-web`; `TestDemandSetIsRegistered` fails if any of them stops being. -56 serve at least one operation, and the one that does not (`rds-data`) is named -with its reason in [coverage.md](coverage.md). Everything below them — support 1 +All 57 serve at least one operation. Everything below them — support 1 and support 0 — is the 226 services that carry no depth commitment. They are registered and engine-served, so a call to one is answered locally rather than billed; what this study declined to promise is that it is answered *faithfully*. diff --git a/docs/faq.md b/docs/faq.md index 3cc90efe..18b2337a 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -38,13 +38,13 @@ For services with a persistent backend (S3, DynamoDB, Lambda, IAM/STS), yes — ## Compatibility **Will my existing boto3 code work?** -Most apps using the core services (S3, SQS, DynamoDB, Lambda, IAM, STS) and common integration services (SNS, CloudWatch, KMS, Secrets Manager, EventBridge, CloudFormation) work with only an `endpoint_url` change. The 1,530-test boto3 suite runs green in CI on every push; what that does and does not promise is spelled out in [compatibility-policy.md](compatibility-policy.md#wire-behaviour--scoped-to-the-compatibility-suite). +Most apps using the core services (S3, SQS, DynamoDB, Lambda, IAM, STS) and common integration services (SNS, CloudWatch, KMS, Secrets Manager, EventBridge, CloudFormation) work with only an `endpoint_url` change. The 1,462-test boto3 suite runs green in CI on every push; what that does and does not promise is spelled out in [compatibility-policy.md](compatibility-policy.md#wire-behaviour--scoped-to-the-compatibility-suite). **What about Terraform / CDK?** Point the AWS provider or CDK at `http://localhost:4747` with dummy credentials. Common resources (`aws_s3_bucket`, `aws_dynamodb_table`, `aws_lambda_function`) work out of the box. Complex IAM policies and deeply CSP-coupled resources are out of scope. **Does it enforce IAM policies?** -Not by default. IAM accepts policy documents for round-trip compatibility but does not evaluate them. Experimental enforcement: `services.iam.enforce_policies: true`. +Not by default. IAM accepts policy documents for round-trip compatibility but does not evaluate them. Policy simulation (`SimulateCustomPolicy`, `SimulatePrincipalPolicy`) unconditionally returns `allowed`. No per-request enforcement is implemented. ## Contributing diff --git a/docs/fidelity-manifest.md b/docs/fidelity-manifest.md index 60ee3ae0..fc44f916 100644 --- a/docs/fidelity-manifest.md +++ b/docs/fidelity-manifest.md @@ -57,8 +57,8 @@ curl -s 'localhost:4747/devcloud/api/fidelity?service=s3' # per-operation, one s { "s3": { "modelBacked": true, - "counts": {"hand-verified": 37, "unimplemented": 70}, - "operations": {"PutObject": "hand-verified", "SelectObjectContent": "unimplemented"} + "counts": {"hand-verified": 43, "auto-crud": 64, "unimplemented": 0}, + "operations": {"PutObject": "hand-verified", "SelectObjectContent": "hand-verified"} } } ``` diff --git a/docs/services-matrix.md b/docs/services-matrix.md index 57d4e9f4..dc2b94ac 100644 --- a/docs/services-matrix.md +++ b/docs/services-matrix.md @@ -40,7 +40,7 @@ These are wired end to end, not stubbed: | Integration | Implementation | |-------------|----------------| -| CloudFormation → 6 resource types | `cloudformation/engine.go` — topological sort, intrinsic functions | +| CloudFormation → 10 resource types | `cloudformation/engine.go` — topological sort, intrinsic functions | | DynamoDB → DynamoDB Streams | Write path publishes records | | DynamoDB Streams → Lambda | `lambda/eventsource.go` polls stream shards | | SQS → Lambda | Event source poller | @@ -57,6 +57,8 @@ These are wired end to end, not stubbed: | REST-JSON | `application/json` | ACM, API Gateway, S3Tables, MWAA, IdentityStore | | REST-XML | `application/xml` | S3, Route53, CloudFront | | Query | `application/x-www-form-urlencoded` | IAM, STS, SQS, SNS, RDS, EC2, AutoScaling | +| EC2 Query | `application/x-www-form-urlencoded` | EC2 (EC2-compliant XML with ``) | +| RPC-v2 CBOR | `application/cbor` | Partner Central Revenue Measurement | SQS speaks both Query and JSON; the protocol is detected per request. diff --git a/docs/services/dynamodb.md b/docs/services/dynamodb.md index 87676890..8f602e8f 100644 --- a/docs/services/dynamodb.md +++ b/docs/services/dynamodb.md @@ -11,7 +11,7 @@ M (Map), and the SS / NS / BS sets. ## Supported APIs -These 20 operations are `hand-verified` — implemented by the provider, not by +These 29 operations are `hand-verified` — implemented by the provider, not by the [CRUD engine](../crud-engine.md). Everything else DynamoDB models is served at a lower tier or not at all; [fidelity-manifest.md](../fidelity-manifest.md) is the per-operation answer. @@ -131,14 +131,9 @@ aws --endpoint-url http://localhost:4747 dynamodb get-item \ `LastEvaluatedKey` and no `ConsumedCapacity`, so `Query`/`Scan` return the whole matching set in one page and code that loops on the cursor sees one iteration. -- **No PartiQL** — `ExecuteStatement`, `ExecuteTransaction` and - `BatchExecuteStatement` are unimplemented and fail rather than answering. -- **Backups, global tables and exports answer from the CRUD engine.** - `CreateBackup`, `CreateGlobalTable`, `DescribeContinuousBackups` and their - neighbours return stored, plausible shapes with no behaviour behind them — - see [crud-engine.md](../crud-engine.md). -- No Kinesis streaming destination (`EnableKinesisStreamingDestination` is - unimplemented) +- **PartiQL subset only** — `ExecuteStatement`, `ExecuteTransaction` and `BatchExecuteStatement` support single-table `SELECT`, `INSERT`, `UPDATE`, and `DELETE`. `WHERE` clauses are limited to simple equality (`=`). No nested paths, subqueries, or arithmetic. +- **Backups and global tables answer from the CRUD engine.** `CreateBackup`, `CreateGlobalTable`, `DescribeContinuousBackups` and their neighbours return stored, plausible shapes with no behaviour behind them. However, `ExportTableToPointInTime`, `ImportTable`, `RestoreTableFromBackup`, and `RestoreTableToPointInTime` are fully handled by the provider (cloning tables and items in SQLite). +- **No Kinesis active replication.** `EnableKinesisStreamingDestination` and `DisableKinesisStreamingDestination` persist configuration, but write events are not actively replicated to Kinesis streams. - No provisioned-throughput accounting or throttling; `BillingMode` is recorded, never enforced - Single account model (account ID: `000000000000`) diff --git a/docs/services/eventbridge.md b/docs/services/eventbridge.md index dad11bd2..de53b7b3 100644 --- a/docs/services/eventbridge.md +++ b/docs/services/eventbridge.md @@ -1,10 +1,30 @@ # EventBridge +## Overview DevCloud supports EventBridge rules and local SQS, SNS and Lambda targets, partner event sources, reusable connections and event bus resource policies. Requests use the standard boto3 `events` client against the DevCloud endpoint. The local account is `000000000000`. +## Supported APIs + +These 45 operations are `hand-verified` — implemented by the provider, not by the CRUD engine. + +| Operation | Description | +|-----------|-------------| +| CreateEventBus / DeleteEventBus / ListEventBuses / DescribeEventBus | Manage custom event buses | +| PutPermission / RemovePermission | Resource-based policies for event buses | +| PutRule / DescribeRule / DeleteRule / ListRules / EnableRule / DisableRule | Event routing rules | +| PutTargets / RemoveTargets / ListTargetsByRule | Manage rule targets (supports local SQS, SNS, and Lambda targets) | +| PutEvents | Dispatch custom events | +| CreatePartnerEventSource / DeletePartnerEventSource / DescribePartnerEventSource / ListPartnerEventSources / ListPartnerEventSourceAccounts / DescribeEventSource / ListEventSources / ActivateEventSource / DeactivateEventSource | Manage partner event sources | +| PutPartnerEvents | Dispatch events from partner sources | +| CreateConnection / DescribeConnection / ListConnections / UpdateConnection / DeleteConnection / DeauthorizeConnection | API destination connection configurations | +| CreateArchive / DescribeArchive / ListArchives / UpdateArchive / DeleteArchive | Event archive lifecycle management | +| StartReplay / DescribeReplay / ListReplays / CancelReplay | Event replay lifecycle | +| TagResource / UntagResource / ListTagsForResource | Resource tagging | +| TestEventPattern | Evaluate event patterns against payloads | + ## Partner event sources `CreatePartnerEventSource` creates a PENDING source. Names follow @@ -81,3 +101,41 @@ The boto3 compatibility suite verifies all four newly supported operations, typed dates, empty Unit responses, connection reuse and real partner delivery to SQS with Unicode and punctuation intact. See [Coverage](../coverage.md) for the whole AWS surface and [Lambda](lambda.md) for Docker runtime requirements. + + +## boto3 Examples + +```python +import boto3 + +client = boto3.client('events', endpoint_url='http://localhost:4747', region_name='us-east-1') + +# Create an event bus and rule +client.create_event_bus(Name='my-bus') +client.put_rule(Name='my-rule', EventBusName='my-bus', EventPattern='{"source": ["my.app"]}') +client.put_targets( + Rule='my-rule', + EventBusName='my-bus', + Targets=[{'Id': '1', 'Arn': 'arn:aws:sqs:us-east-1:000000000000:my-queue'}] +) + +# Dispatch an event +client.put_events(Entries=[{ + 'Source': 'my.app', + 'DetailType': 'test', + 'Detail': '{"key": "value"}', + 'EventBusName': 'my-bus' +}]) +``` + +## AWS CLI Examples + +```bash +aws --endpoint-url=http://localhost:4747 events create-event-bus --name my-bus +aws --endpoint-url=http://localhost:4747 events put-events --entries '[{"Source": "my.app", "DetailType": "test", "Detail": "{\"key\": \"value\"}", "EventBusName": "my-bus"}]' +``` + +## Known Limitations +- **External targets are not supported**: Target delivery is restricted to DevCloud's own SQS, SNS, and Lambda services. +- **Connection credentials (`OAuth`)**: Credentials are stored but not actively authorized with external identity providers. +- **Bus policies**: Policies are stored for round-trip compatibility but not actively evaluated by IAM. diff --git a/docs/services/lambda.md b/docs/services/lambda.md index e4016fe5..fc8faebe 100644 --- a/docs/services/lambda.md +++ b/docs/services/lambda.md @@ -29,7 +29,7 @@ and `us-east-1`. Explicit function environment values may override these default ## Supported APIs -These 25 operations are hand verified at the scope exercised by the compatibility +These 35 operations are hand verified at the scope exercised by the compatibility suite. Unlisted operations use the documented fidelity tiers. | Operation | Behaviour | @@ -103,7 +103,7 @@ preserve version/alias references and log delivery failures. ## Limits and verification -No layers, function URLs, reserved/provisioned concurrency, policy enforcement, +No runtime layer mounting (layers can be managed, but not attached to function execution), function URLs, reserved/provisioned concurrency, policy enforcement, SQS partial batch responses, FIFO/DLQ parity or exactly-once delivery are promised. ZIP extraction rejects traversal, symlinks, special files and reserved adapter names; limits are 10,000 entries and 250 MiB uncompressed. @@ -119,3 +119,6 @@ Opt-in requires both images and a reachable daemon; missing dependencies fail instead of skipping. Native Colima test runs can set `DEVCLOUD_LAMBDA_CALLBACK_HOST=host.lima.internal`. Skipped runtime tests are not execution evidence. + +- **Streaming invocations** do not use binary EventStream frames; they run synchronously and return the full JSON payload at once with chunked headers. +- **Durable executions** track state in SQLite but there is no workflow orchestrator or automatic resume functionality. diff --git a/docs/services/s3.md b/docs/services/s3.md index 296d95c0..b7966fc8 100644 --- a/docs/services/s3.md +++ b/docs/services/s3.md @@ -6,7 +6,7 @@ DevCloud S3 provides basic object storage using the filesystem for object data a ## Supported APIs -These 40 operations are `hand-verified` — implemented by the provider, not by +These 43 operations are `hand-verified` — implemented by the provider, not by the [CRUD engine](../crud-engine.md). A sub-resource this provider does not serve (`?lifecycle`, `?encryption`, …) returns a clean AWS error rather than being quietly answered as a bucket listing; diff --git a/docs/services/sns.md b/docs/services/sns.md index a5b315ad..61e846df 100644 --- a/docs/services/sns.md +++ b/docs/services/sns.md @@ -1,10 +1,30 @@ -# SNS +# SNS (Simple Notification Service) +## Overview DevCloud supports topics/subscriptions, batch publication to local SQS, FIFO admission, platform application/endpoint settings, SMS settings, opt-in and SMS sandbox verification. Use the standard boto3 sns client against the local endpoint. The local account is `000000000000` and region is `us-east-1`. +## Supported APIs + +These 41 operations are `hand-verified` — implemented by the provider, not by the CRUD engine. + +| Operation | Description | +|-----------|-------------| +| CreateTopic / DeleteTopic / ListTopics | Topic lifecycle | +| GetTopicAttributes / SetTopicAttributes | Topic configurations (including `.fifo` topics) | +| Subscribe / Unsubscribe / ListSubscriptions / ListSubscriptionsByTopic / ConfirmSubscription | Subscription lifecycle | +| GetSubscriptionAttributes / SetSubscriptionAttributes | Subscription configurations | +| Publish / PublishBatch | Message publication, including FIFO deduplication and ordering | +| CreatePlatformApplication / DeletePlatformApplication / ListPlatformApplications / GetPlatformApplicationAttributes / SetPlatformApplicationAttributes | Mobile push application configurations | +| CreatePlatformEndpoint / DeleteEndpoint / ListEndpointsByPlatformApplication / GetEndpointAttributes / SetEndpointAttributes | Mobile push endpoints | +| SetSMSAttributes / GetSMSAttributes / OptInPhoneNumber / CheckIfPhoneNumberIsOptedOut / ListPhoneNumbersOptedOut | SMS global settings | +| CreateSMSSandboxPhoneNumber / VerifySMSSandboxPhoneNumber / ListSMSSandboxPhoneNumbers / DeleteSMSSandboxPhoneNumber / GetSMSSandboxAccountStatus | SMS sandbox and OTP generation | +| AddPermission / RemovePermission | Resource-based topic policies | +| TagResource / UntagResource / ListTagsForResource | Resource tagging | +| PutDataProtectionPolicy / GetDataProtectionPolicy | Data protection policies | + ## Publication and FIFO Publish and PublishBatch share validation and delivery. Batches contain 1–10 @@ -99,3 +119,37 @@ challenge; issue a fresh OTP before verification. Pinned boto3 tests exercise Query/XML responses and actual SQS/outbox results. Invalid parameters, resource absence, verification and storage/delivery failures use distinct errors. A closed database does not produce an empty successful list. + + +## boto3 Examples + +```python +import boto3 + +client = boto3.client('sns', endpoint_url='http://localhost:4747', region_name='us-east-1') + +# Standard topic creation and subscription +topic = client.create_topic(Name='my-topic') +client.subscribe( + TopicArn=topic['TopicArn'], + Protocol='sqs', + Endpoint='arn:aws:sqs:us-east-1:000000000000:my-queue' +) + +# Publish a message +client.publish( + TopicArn=topic['TopicArn'], + Message='hello world' +) +``` + +## AWS CLI Examples + +```bash +aws --endpoint-url=http://localhost:4747 sns create-topic --name my-topic +aws --endpoint-url=http://localhost:4747 sns publish --topic-arn arn:aws:sns:us-east-1:000000000000:my-topic --message "hello" +``` + +## Known Limitations +- **Delivery endpoints:** Active delivery is only implemented for SQS (`sqs` protocol) and Lambda (`lambda` protocol). HTTP/S, email, and mobile push endpoints are accepted as configuration but no outbound network requests are made. +- **SMS Delivery:** SMS messages are not dispatched to real telecommunication networks. OTP codes for the SMS sandbox are generated and stored locally in the `sns.db` outbox table for verification (see Sandbox section). diff --git a/hugo.toml b/hugo.toml index 3136f59a..69f8cd33 100644 --- a/hugo.toml +++ b/hugo.toml @@ -102,6 +102,16 @@ enableGitInfo = true parent = 'services-matrix' pageRef = '/services/iam-sts' weight = 85 + [[menu.before]] + identifier = 'eventbridge' + parent = 'services-matrix' + pageRef = '/services/eventbridge' + weight = 86 + [[menu.before]] + identifier = 'sns' + parent = 'services-matrix' + pageRef = '/services/sns' + weight = 87 [[menu.before]] identifier = 'demand'