[SVLS-9800] fix(config): Support AWS Secrets Manager ARNs for dual-shipping additional endpoints

[shreyamalpani]

Summary

Adds support for sourcing dual-shipping to additional endpoints (across metrics, traces, and logs) from an AWS Secrets Manager secret ARN, instead of requiring the endpoints/keys to be embedded as plaintext env vars. This mirrors the existing pattern for resolving the primary DD_API_KEY from Secrets Manager.

Three new env vars are read at startup and, if set, resolved and merged into the existing plaintext additional-endpoints config:

• DD_ADDITIONAL_ENDPOINTS_SECRET_ARN → merges into DD_ADDITIONAL_ENDPOINTS (metrics)
• DD_APM_ADDITIONAL_ENDPOINTS_SECRET_ARN → merges into DD_APM_ADDITIONAL_ENDPOINTS (traces)
• DD_LOGS_CONFIG_ADDITIONAL_ENDPOINTS_SECRET_ARN → merges into DD_LOGS_CONFIG_ADDITIONAL_ENDPOINTS (logs)

Secret content uses the same JSON shape as the corresponding plaintext env var as defined in docs.

Motivation

GitHub issue #1370

Design

• Resolution is done synchronously at startup, not lazily on first flush. Since a non-empty ARN means the fetch will happen unconditionally, there's no benefit to deferring it, and doing it eagerly avoids adding latency to flush-time paths.
• Resolved secrets overwrite the corresponding plaintext additional-endpoints config, when the secret's ARN is set and resolves successfully. If the ARN is unset, the fetch fails, or the secret fails to parse, the plaintext value is left untouched — it's not merged with the resolved value.
• No changes to the existing primary DD_API_KEY resolution path (create_api_key_factory/resolve_secrets) — this is a separate, additive resolution step.
• Also bumps the pinned serverless-components git rev to 8156757 to pick up the corresponding datadog-agent-config schema changes (new *_secret_arn fields) from config: add additional endpoints secret ARN environment variables serverless-components#171

Testing

• Manually tested and verified dual shipping works for metrics, logs, and traces to an additional endpoint sourced from a Secrets Manager ARN

[datadog-prod-us1-3]

🔄 Datadog auto-retried 1 job - 0 passed on retry

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 968d01c | Docs | View more details | Give us feedback!}

[Copilot]

Copilot review overview

🟡 Changes recommended

Add bounded timeouts for startup secret requests and broader feature-path test coverage.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 · 1

Open (2)

• Add timeout for Secrets Manager startup fetches · New
• Add async Secrets Manager success and failure path coverage · New

What changed in this PR

Adds AWS Secrets Manager ARN support for dual-shipping metrics, traces, and logs endpoint configuration.

Changes:

• Resolves endpoint secrets during startup with plaintext fallback on failure.
• Integrates resolution before telemetry flushers initialize.
• Updates serverless-components dependency revisions.

File	Description
bottlecap/​src/​secrets/​decrypt.rs	Implements secret fetching, parsing, and endpoint resolution.
bottlecap/​src/​bin/​bottlecap/​main.rs	Integrates endpoint resolution at startup.
bottlecap/​Cargo.toml	Updates dependency revision.
bottlecap/​Cargo.lock	Locks updated dependencies.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.