diff --git a/.github/workflows/deploy.yml b/.github/workflows/deploy.yml index ebdb805..8d02654 100644 --- a/.github/workflows/deploy.yml +++ b/.github/workflows/deploy.yml @@ -15,13 +15,18 @@ jobs: - uses: actions/checkout@v4 with: fetch-depth: 0 + # package.json sets "packageManager": "yarn@4.18.0", which the runner's + # bundled Yarn 1 refuses to run. Corepack must be enabled before + # setup-node so its yarn cache detection sees Yarn 4, not Yarn 1. + - name: Enable Corepack + run: corepack enable - uses: actions/setup-node@v4 with: - node-version: 18 + node-version: 22 cache: yarn - name: Install dependencies - run: yarn install --frozen-lockfile + run: yarn install --immutable - name: Build website run: yarn build diff --git a/.github/workflows/test-deploy.yml b/.github/workflows/test-deploy.yml index c22bcea..b7f3cb1 100644 --- a/.github/workflows/test-deploy.yml +++ b/.github/workflows/test-deploy.yml @@ -15,12 +15,17 @@ jobs: - uses: actions/checkout@v4 with: fetch-depth: 0 + # package.json sets "packageManager": "yarn@4.18.0", which the runner's + # bundled Yarn 1 refuses to run. Corepack must be enabled before + # setup-node so its yarn cache detection sees Yarn 4, not Yarn 1. + - name: Enable Corepack + run: corepack enable - uses: actions/setup-node@v4 with: - node-version: 18 + node-version: 22 cache: yarn - name: Install dependencies - run: yarn install --frozen-lockfile + run: yarn install --immutable - name: Test build website run: yarn build diff --git a/.gitignore b/.gitignore index b2d6de3..b52f048 100644 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,17 @@ # Dependencies /node_modules +# Yarn Berry. install-state.gz is a local install cache that is rewritten on +# every `yarn install`, so committing it churns ~1.8MB of binary per install. +# https://yarnpkg.com/getting-started/qa#which-files-should-be-gitignored +.yarn/* +!.yarn/patches +!.yarn/plugins +!.yarn/releases +!.yarn/sdks +!.yarn/versions +.pnp.* + # Production /build diff --git a/.yarnrc.yml b/.yarnrc.yml new file mode 100644 index 0000000..063ed60 --- /dev/null +++ b/.yarnrc.yml @@ -0,0 +1,8 @@ +approvedGitRepositories: + - "**" + +enableScripts: true + +nodeLinker: node-modules + +npmMinimalAgeGate: 0 diff --git a/README.md b/README.md index aaba2fa..860f35c 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # Website -This website is built using [Docusaurus 2](https://docusaurus.io/), a modern static website generator. +This website is built using [Docusaurus](https://docusaurus.io/), a modern static website generator. ### Installation @@ -24,6 +24,43 @@ $ yarn build This command generates static content into the `build` directory and can be served using any static contents hosting service. +### The install script + +`static/install` is the protoconf installer that the docs point users at: + +``` +curl -s https://protoconf.dev/install | sh +curl -s https://protoconf.dev/install | sh -s -- protoconf-terraform +``` + +It installs one of the packages listed in its `select_package` function — +today `protoconf` and `protoconf-terraform`, named as the one positional +argument and defaulting to `protoconf`. Adding another means adding a case +there with the repository, the platforms that repository builds, and a docs +URL; nothing else in the script is package-specific. + +Docusaurus copies `static/` verbatim into `build/`, so the file is served at +`/install` with no build step of its own. Two things it depends on: + +- **The apex domain has to serve the site directly.** The one-liner above uses + `curl -s` without `-L`, so a redirect from `protoconf.dev` to + `www.protoconf.dev` would pipe an empty body into `sh` and silently install + nothing. Attach both hostnames to the same deployment rather than redirecting + one to the other. +- **`static/_headers`** sets `Content-Type: text/plain` for `/install` on + Cloudflare, so the URL is readable in a browser instead of downloading. + +The script takes the release archives published by goreleaser, so the platform +list for each package has to stay in step with the build matrix in that +repository's `.goreleaser.yaml` — they are not the same today: protoconf +publishes a `linux/386` build and protoconf-terraform does not. Test a change +to it end to end, against every package, before merging: + +``` +sh static/install --dir /tmp/protoconf-install-test +sh static/install protoconf-terraform --dir /tmp/protoconf-install-test +``` + ### Deployment Using SSH: diff --git a/blog/2023-05-16-new-site.mdx b/blog/2023-05-16-new-site.mdx index 158c77b..672fde2 100644 --- a/blog/2023-05-16-new-site.mdx +++ b/blog/2023-05-16-new-site.mdx @@ -9,6 +9,8 @@ Hello to all our amazing users, We are excited to announce the launch of our new documentation site! We've been hard at work behind the scenes to make it easier for you to find the information you need. +{/* truncate */} + The new site is more than just a fresh coat of paint. It's a fundamental rethinking of how we present information about our product. Our goal is to provide you with clear, concise, and complete information about how to use our services. Here are some of the new features and improvements we've made: diff --git a/blog/2026-09-03-protoconf-0-2-0.mdx b/blog/2026-09-03-protoconf-0-2-0.mdx new file mode 100644 index 0000000..ca4dfeb --- /dev/null +++ b/blog/2026-09-03-protoconf-0-2-0.mdx @@ -0,0 +1,118 @@ +--- +slug: protoconf-0-2-0 +title: Protoconf v0.2.0 +authors: [smintz] +tags: [protoconf, release] +--- + +Protoconf v0.2.0 is out — the first feature release since v0.1.7. It brings +remote proto modules, staged rollouts, a Kubernetes ConfigMaps backend, one-shot +config reads over gRPC and plain HTTP, opt-in OpenTelemetry, a formatter, and a +single consistent configuration story across every command. + +{/* truncate */} + +## Remote modules + +Configuration repositories can now depend on protos and Starlark that live +somewhere else. Declare the dependency in a `CONFIGSPACE` file, pin it to a tag, +branch, commit or checksum, and let `protoconf mod tidy` resolve it into +`protoconf.lock` and a local cache: + +```python title=CONFIGSPACE +platform_protos = remote_repo( + url = "github.com/acme/platform-protos", + tag = "v1.4.0", +) +``` + +```python title=./src/myproject/server_config.pconf +load("@platform_protos//platform/v1/service.proto", "Service") +``` + +The lock file is what makes a compile reproducible — commit it next to your +configs. [Remote Modules →](/docs/0.2.0/advanced-usage/modules) + +## Staged rollouts + +A configuration change is a deploy, and v0.2.0 lets you treat it like one. Wrap +a config in a `ConfigRollout` and it reaches a canary channel first, then a +percentage of your fleet, with a cooldown between each stage: + +```python +def main(): + return ConfigRollout( + value = ServerConfiguration(max_connections = 2000), + default_cooldown_time = Duration(seconds = 300), + stages = [ + RolloutStage(channel = "canary"), + RolloutStage(percentile = 10), + RolloutStage(percentile = 50), + ], + ) +``` + +Agents started with `-enable-rollout` pick the first stage they match, hashing +their own identity so the same host lands in the same bucket every time. +Interrupting `protoconf insert` mid-rollout clears the plan and returns every +agent to the stable config. [Staged Rollouts →](/docs/0.2.0/production/rollouts) + +## Kubernetes ConfigMaps + +`configmaps` is a new key-value backend for both the agent and the inserter, so +a cluster can distribute protoconf configs without running Consul, etcd or +ZooKeeper alongside it: + +```bash +protoconf agent -store configmaps -namespace protoconf +``` + +[Running on Kubernetes →](/docs/0.2.0/production/kubernetes) + +## One-shot reads, including over HTTP + +Not every consumer is a long-running service. `GetConfig` is the unary +counterpart to `SubscribeForConfig`, for jobs, scripts and init containers. The +agent also transcodes it onto its admin HTTP listener, so anything that can make +an HTTP request can read a config: + +```bash +curl http://localhost:4380/v1/config/myproject/server_config +``` + +[Consuming Configuration Updates →](/docs/0.2.0/consume-config-updates#one-shot-reads) + +## Telemetry you opt into + +OpenTelemetry export is now off by default and enabled with `-enable-otel`. An +agent that is not asked to export traces installs no-op providers and contacts +nothing. Prometheus metrics stay where they were, on `/metrics`, no flag +required. [Observability →](/docs/0.2.0/production/observability) + +## `protoconf fmt` + +One canonical layout for `.pconf`, `.mpconf`, `.pinc`, `.proto-validator` and +`.star` files, and a `-l` mode to fail CI when something is unformatted: + +```bash +protoconf fmt -w . +``` + +[Formatting →](/docs/0.2.0/advanced-usage/formatting) + +## Two things to check when upgrading + +**Configuration precedence.** Every command now resolves configuration as +`flags > environment variables > config file > proto defaults`. `protoconf +agent` used to let a config file override `PROTOCONF_AGENT_*` environment +variables; it no longer does. With repeated `-config-file` flags, the later file +now wins. + +**Telemetry is off by default.** An agent that used to reach your collector +automatically needs `-enable-otel` now. + +The full list is in +[What's New in v0.2.0](/docs/0.2.0/whats-new-0.2.0), and the complete changelog +is on the [release page](https://github.com/protoconf/protoconf/releases/tag/v0.2.0). + +As always, we're on [Discord](https://discord.protoconf.sh) if you hit anything. diff --git a/docs/advanced-usage/_category_.json b/docs/advanced-usage/_category_.json index 97d1339..d160208 100644 --- a/docs/advanced-usage/_category_.json +++ b/docs/advanced-usage/_category_.json @@ -1,8 +1,8 @@ { "label": "Advanced Usage", - "position": 5, + "position": 6, "link": { "type": "generated-index", "description": "Learn more advanced techniques to use protoconf." } -} \ No newline at end of file +} diff --git a/docs/advanced-usage/formatting.mdx b/docs/advanced-usage/formatting.mdx new file mode 100644 index 0000000..cc394e9 --- /dev/null +++ b/docs/advanced-usage/formatting.mdx @@ -0,0 +1,60 @@ +--- +id: formatting +title: Formatting Starlark Sources +sidebar_position: 6 +--- + +# Formatting Starlark Sources + +:::info New in v0.2.0 +::: + +`protoconf fmt` formats protoconf's Starlark sources the way `gofmt` formats +Go — one canonical layout, no configuration, no debate in code review. + +It handles every Starlark extension protoconf uses: + +- `.pconf` — configuration files +- `.mpconf` — multi-output configuration files +- `.pinc` — shared includes +- `.proto-validator` — validation rules +- `.star` — plain Starlark + +## Usage + +```bash +# Print the formatted result of any file that is not already formatted, +# without changing anything on disk +protoconf fmt + +# Rewrite files in place +protoconf fmt -w . + +# Show what would change +protoconf fmt -d . + +# List only the files that are not formatted — useful in CI +protoconf fmt -l . +``` + +With no path arguments, `protoconf fmt` walks the current directory +recursively. You can also pass individual files or directories: + +```bash +protoconf fmt -w src/myproject/server_config.pconf src/shared/ +``` + +## In continuous integration + +`-l` prints the files whose formatting differs and is the check to run in CI: + +```yaml title=.github/workflows/protoconf.yml +- name: Check formatting + run: | + unformatted="$(protoconf fmt -l .)" + if [ -n "$unformatted" ]; then + echo "These files need 'protoconf fmt -w':" + echo "$unformatted" + exit 1 + fi +``` diff --git a/docs/advanced-usage/modules.mdx b/docs/advanced-usage/modules.mdx new file mode 100644 index 0000000..fc80746 --- /dev/null +++ b/docs/advanced-usage/modules.mdx @@ -0,0 +1,112 @@ +--- +id: modules +title: Remote Modules +sidebar_position: 5 +--- + +# Remote Modules + +:::info New in v0.2.0 +::: + +A configuration repository rarely owns every schema it needs. Remote modules +let a protoconf repository depend on `.proto` and Starlark files that live in +another repository or archive, pinned to an exact revision and cached locally — +the same shape as Go modules or Bazel external repositories. + +## Declaring dependencies + +Dependencies are declared in a file named `CONFIGSPACE` at the root of your +protoconf directory. It is a Starlark file with one builtin, `remote_repo`: + +```python title=CONFIGSPACE +platform_protos = remote_repo( + url = "github.com/acme/platform-protos", + tag = "v1.4.0", +) + +vizceral_repo = remote_repo( + label = "vizceral_repo", + url = "./internal/vizceral.tgz", + checksum = "896b13d56bd1787089ca5767656c7ef1", +) +``` + +Each `remote_repo` accepts: + +| Argument | Description | +| --- | --- | +| `url` | Where to fetch from — a Git repository, an archive URL, or a local path | +| `tag` / `branch` / `commit` / `checksum` | How the dependency is pinned. Provide exactly one | +| `label` | The name used to reference the module. Derived from the URL when omitted | +| `source_path` | Directory inside the repository holding the protos. Defaults to `src` | +| `additional_proto_dirs` | Extra directories to add to the proto import path | +| `exclude_file_regexps` | Patterns for files to skip | + +Pinning is not optional in practice: a `remote_repo` with no `tag`, `branch`, +`commit` or `checksum` resolves against a moving target, and protoconf prints a +warning telling you so. + +## Resolving and downloading + +`protoconf mod` turns `CONFIGSPACE` into a lock file and a local cache: + +```bash +# Resolve CONFIGSPACE into protoconf.lock +protoconf mod init + +# Download everything protoconf.lock pins into .protoconf_cache/ +protoconf mod sync + +# Both, in one step — the usual command +protoconf mod tidy +``` + +`protoconf.lock` records the resolved fetch URL and content hashes for every +dependency, including transitive ones: + +```json title=protoconf.lock +{ + "url": ".", + "deps": { + "vizceral_repo": { + "label": "vizceral_repo", + "url": "./internal/vizceral.tgz", + "checksum": "896b13d56bd1787089ca5767656c7ef1", + "sourcePath": "src", + "fileDescriptorSetSum": "039f1e1023250b34054894cc58bc8b2b" + } + } +} +``` + +Commit `protoconf.lock` alongside your configs — it is what makes a compile +reproducible. The `.protoconf_cache/` directory is derived and belongs in +`.gitignore`. + +## Loading from a module + +Inside a `.pconf`, `.mpconf` or `.pinc` file, reference a module with +`@