Skip to content

feat: the two-level product-root layout, and a schema-driven dev environment - #25

Merged
justinmerrell merged 3 commits into
mainfrom
feat/layout-product-root
Sep 22, 2026
Merged

justinmerrell merged 3 commits into
mainfrom
feat/layout-product-root

Conversation

@justinmerrell

Copy link
Copy Markdown
Contributor

Two halves of one idea: a repository has two levels, and each level declares the environment it reads.

1. The layout pattern

LAYOUT.md states the rule host-agent adopted in #23: the repository root holds what acts on the product; one directory named after the repo holds what it is built from. It covers the placement test, the invariants and which code enforces each, the per-ecosystem adapters (Rust proven, the rest marked "verify on first adoption"), the compatibility gate for tools that assume the root, adoption steps, and an Evidence section — every citation re-opened and confirmed.

.repo/layout.toml declares the product. The template ships product = "", so product-dependent rules skip while the rule that the root stays free of product content still applies.

New policies, all inside repo check:

Policy Enforces
layout (LAYOUT-01..11) No manifest, lockfile, toolchain file or source tree at the root; the declared dir exists, is named after the repo, has its manifest and env.schema.yaml; devcontainer mounts, editor links, Dependabot and PRODUCT_DIR agree with the declaration
paths (PATH-01..04) Every lefthook glob, .gitattributes pattern, paths-filter, working-directory, Dependabot directory and Taskfile path var still names something — the generic half of host-agent's check-path-refs.py
env (ENV-01..07) See below
config (new CFG-09) A .config/ path a caller names must exist (CFG-04 only checked the other direction)

The product env contract lives at <product>/env.schema.yaml, not under config/. LAYOUT-11 fails the old placement and its fix names the root location.

2. The dev environment reads a schema too

.devcontainer/env.schema.yaml replaces the hand-written .env.example, which becomes a generated artifact (task env:render, ENV-02 fails on drift). The rendering has to exist: docker run --env-file and Compose read dotenv, and initializeCommand runs on the host where only bash is guaranteed. No standard removes that step — varlock/@env-spec, dotenvx, Compose and the devcontainer spec were all checked, and platform keeps a hand-written file for the same reason.

What the schema buys beyond documentation:

  • Requiredness follows the enabled profiles. consumers: names the stacks that read a binding, so a value is only demanded while one of them runs.
  • A missing value never blocks the container. startup.sh starts every stack that can and skips the rest by name; the MOTD repeats it; task env:setup asks which stacks you want first, then only for what that answer makes necessary — masked secrets, menus for fixed choices, the binding's description as the prompt.
  • source: host fills from the host environment through a bash-readable # @host marker, and mirrors into devcontainer.json secrets so Codespaces prompts (ENV-06).
  • mirrored_in: turns the Tempo/Loki credential duplication from a note in the docs into a check (ENV-05).
  • Edits apply without a rebuild. The shell profile re-loads .env (lib/env-load.sh parses rather than sources, because dotenv is not shell), and startup.sh exports COMPOSE_PROFILES from the file so the copy runArgs --env-file froze at docker run cannot win.

This retires scripts/lib/env-check.sh and the CI job that duplicated it, with its ruleset entry and hooks allowlist. The generated .env.example parses to exactly the keys and values the old one had, so no existing .env changes meaning.

Template bugs this surfaced

  • Two .gitattributes lines matched no tracked file.
  • The pre-commit governance job's glob skipped .repo/**, so editing a policy never re-ran the checks.
  • task repo:install reused uv's cached build, so an edited policy never took effect.
  • The comments policy and codespell scanned .repo/.venv.
  • COMPOSE_PROFILES= was flagged as an unfilled required key on every boot.

Verification

  • repo check: every policy green. 90 pytest cases (.repo/tests/, new CI step) — most layout rules only fire once a product is declared, which the template never does.
  • Mutation smoke tests on a copy of this repo: a root Cargo.toml, a declared-but-missing product, a stray schema, the .env gitattributes line, a mistyped config path and a renamed LAYOUT.md heading each fail with the expected code.
  • End-to-end on a copy with a required binding added: host-side fill works, doctor blocks only redis while minio still starts, setup's menu enables a profile and then asks for what it made necessary, and the loader exports a value containing spaces correctly.
  • ShellCheck, markdownlint, yamllint, codespell and actionlint at their pinned versions; docker compose config with every profile enabled.
  • Downstream: against a fresh host-agent clone, layout, paths and env all pass after only renaming WORKSPACE_DIR to PRODUCT_DIR. That run caught a false positive in LAYOUT-08 (host-agent's release scripts legitimately carry their own uv project), now narrowed to updates scanning /.

Follow-ups

  • host-agent adopts layout.toml + PRODUCT_DIR, then retires the generic half of check-path-refs.py.
  • Upstream repo env to platform, whose devcontainer .env.example is still hand-written.
  • A multi-product variant if a second repo needs one.

🤖 Generated with Claude Code

justinmerrell and others added 3 commits September 22, 2026 16:29
Make the layout host-agent adopted in #23 the pattern every repo scaffolded
from this template inherits: the repository root holds what acts on the
product, one directory named after the repo holds what it is built from.

- LAYOUT.md: the rule, the placement test, invariants, ecosystem adapters,
  the integration-compatibility gate, adoption steps, and the evidence
  (release-please#1724, flox#3997, cryptography#11836, zksync-era#3456).
- .repo/layout.toml declares the product (empty here: the template ships none).
- New policies: `layout` (LAYOUT-01..11: no product content at the root;
  mounts, Dependabot and PRODUCT_DIR agree with the declaration; the env
  contract lives at <product>/env.schema.yaml, never under config/),
  `paths` (PATH-01..04, the generic half of host-agent's check-path-refs.py),
  `env` (ENV-01, the shared env.schema.yaml shape), and CFG-09 (a caller's
  .config/ path must exist).
- A pytest suite, run in CI's governance job: most layout rules only fire
  once a product is declared, which the template never does.

Fixes surfaced along the way: the two .gitattributes lines matching no
tracked file; the governance hook's glob skipping .repo/**; `repo:install`
reusing uv's cached build so policy edits never took effect; the comments
policy and codespell scanning .repo/.venv.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replace the hand-written .env.example with .devcontainer/env.schema.yaml, the
same shape products use for their runtime contract. The rendering stays,
because `docker run --env-file` and Compose read dotenv and the host-side hook
has only bash -- so it becomes a generated artifact that CI checks, not a file
anyone edits.

What the schema buys beyond documentation:

- Requiredness follows the enabled COMPOSE_PROFILES: `consumers:` names the
  stacks that read a binding, and a value is only demanded while one of them
  is running.
- A missing value never blocks the container. startup.sh starts every stack
  that can start and skips the rest, naming them; the MOTD repeats it.
- `task env:setup` asks which stacks to run first, then only for what that
  answer makes necessary -- masked secrets, menus for fixed choices, the
  binding's own description as the prompt.
- `source: host` fills from the host environment (a bash-readable `# @host`
  marker) and mirrors into devcontainer.json `secrets`, so Codespaces prompts.
- `mirrored_in:` makes the Tempo/Loki credential duplication a check (ENV-05)
  rather than a note in the docs.
- The shell profile re-loads .env (lib/env-load.sh, which parses rather than
  sources), so an edited value reaches new terminals without a rebuild, and
  startup.sh exports COMPOSE_PROFILES from the file so the stale copy
  `runArgs --env-file` froze at `docker run` cannot win.

ENV-02..07 check the rendering, compose parity in both directions, the
mirrored literals, the secrets block, and that no secret carries a committed
default. That retires .devcontainer/scripts/lib/env-check.sh and the CI job
that duplicated it, with its ruleset entry and hooks allowlist.

The generated .env.example parses to exactly the keys and values the previous
hand-written one had, so no existing .env changes meaning.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The Feature's default distro ("ms", Microsoft OpenJDK) is gone from SDKMAN's
index -- `sdk list java` now returns no -ms builds at all -- so the pinned
version 17 resolves to nothing and the image build fails with:

    JDK version 17 is available in ms...
    Version 17 not found. Available versions:
    ERROR: Feature "Java (via SDKMAN!)" failed to install!

Unrelated to the layout work: this is the first CI run on the repo since
2026-08-19, so an upstream change surfaced here. Temurin (Eclipse Adoptium)
carries 17 and is the conventional default elsewhere.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@justinmerrell

Copy link
Copy Markdown
Contributor Author

The two container-build jobs failed on something this PR did not touch: the Java Feature's default distro (ms, Microsoft OpenJDK) has disappeared from SDKMAN's index, so the pinned "version": "17" resolves to nothing (Version 17 not found. Available versions: followed by nothing). sdk list java currently returns no -ms builds at all. This branch is the first CI run on the repo since 2026-08-19, which is why it surfaced here. Pinned jdkDistro: tem (Temurin) to unblock.

@justinmerrell
justinmerrell merged commit 4e1d506 into main Sep 22, 2026
6 checks passed
@justinmerrell
justinmerrell deleted the feat/layout-product-root branch September 22, 2026 18:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant