serve: enforce the permission firewall on every REST route (#655)

[padak]

Fixes #655.

The gap

PermissionEngine was built only in the Typer callback (cli.py), so a persisted permissions set --mode deny policy — and both --deny-writes and --deny-destructive — protected the CLI process and nothing else. kbagent serve exposed all 236 routes, DELETE /storage/buckets and POST /token/{p}/delete included, behind a single all-or-nothing bearer token.

#677 built the enforcement machinery — an engine on app.state, a PermissionDeniedError → HTTP 403 handler, and the require_permission dependency — and wired it to three /auth/* routes. This PR adds the missing half: coverage.

How

One app-level dependency (FastAPI(dependencies=[...]), so no router can be added outside it) runs on every request, looks the matched route up by (method, path template) in a central table, and calls the same PermissionEngine.check_or_raise the CLI uses. FastAPI puts the matched route into request.scope["route"] before dependencies resolve, so the template — /storage/buckets/{project}, not the concrete URL — is what the policy is keyed on. Verified at runtime, not assumed.

Why one table instead of 226 per-route decorators:

1. One auditable screen. "What can a caller still do under --deny-destructive?" is answered by reading one file, not thirty routers.
2. Fail-closed by construction. A route with no entry is refused, not silently allowed — the failure mode of a forgotten annotation is a loud 403, never an open door.

The per-route require_permission(...) form still wins where it is declared: such a route is skipped by the table lookup (recognised via a PERMISSION_DEPENDENCY_MARKER attribute). That is what keeps #677's three /auth/* routes — and the test-only probe routes in tests/test_server_permissions.py, registered after create_app — working unchanged.

With no policy configured, nothing changes. is_allowed returns True for everything, so the default install behaves exactly as before.

Also in scope

• GET /permissions/show (the LOW ask in serve: missing mirrors — describe-batch (inline payload), unload-table; job run lacks idempotency_key #657, and serve: PermissionEngine firewall is not enforced on REST routes #655's option-2 discoverability point). Reports the effective policy — the persisted block already merged with the --deny-* flags the daemon was launched with, which a REST caller can neither see nor change. Reachable under any policy (the same anti-lockout rule the CLI has), still requires the bearer token. Deliberately no write counterpart: letting a bearer token widen the policy that constrains it would make the firewall self-defeating, so permissions set / reset stay terminal actions on the host.
• Two new serve-only operations, ai.chat and workspace.sql-improve, both write: neither touches Keboola, but both spawn a local claude/codex/gemini process on the host, exactly like the already-write agent.prompt-improve. Added to SERVE_ONLY_OPERATIONS so the command-sync gate does not read them as dead keys.
• The false comment serve: PermissionEngine firewall is not enforced on REST routes #655 called out. permissions.py's http.* block claimed "The serve's own routes enforce their own permissions on top." It is true now, and the comment says precisely since when.

Classification notes

• Every DELETE route maps to a destructive- or admin-class operation; a test asserts it.
• Eight POSTs map to read operations (/flows/validate, /lineage/show, /kai/ask, …) — POST because the request needs a body, not because it mutates. Explicit allowlist in the test, so a new mutating route classified read fails.
• GET /branches/{project}/merge-url maps to the write-class branch.merge. That is CLI parity, not a slip: kbagent branch merge only ever produces a URL too.
• Granularity caveat, documented: POST /semantic-layer/items/{kind} covers metric/dataset/… in one route, so it maps to the collapsed parent key semantic-layer.add, not semantic-layer.add.metric. A policy naming only a leaf key is enforced on the CLI but not over REST — name the parent or a cli:* category to cover both.
• Nine bootstrap paths are never checked (/health/ping, /health/auth-info, /ui-config, /docs, /redoc, /openapi.json, /docs/oauth2-redirect, and the SPA shell / + /index.html). A locked-down server must still be able to say who it is, or a client cannot tell a policy refusal from a dead process.

Testing

make check green: 6148 passed, 12 skipped. ty clean (the one remaining diagnostic is the pre-existing scripts/hatch_build.py unresolved import). Lint, format, command-sync, version-gates, sentinel-guards, error-codes and the endpoint-reference gate all pass; docs/web-server-endpoints.md regenerated and committed.

31 new tests in tests/test_server_route_permissions.py (new file, so no conflict with #681's edits to test_server_permissions.py):

• Completeness, both directions — every live route is mapped, exempt, or inline-guarded; no table entry matches a dead route; every table value is a real OPERATION_REGISTRY key. This is what keeps the runtime fail-closed branch unreachable in a released build.
• Enforcement on real routes — cli:destructive denies DELETE /storage/buckets/{project} while GET /projects stays 200; --deny-writes denies POST /jobs/{p}/run; an exact token.delete pattern works; mode=deny blocks an unlisted read but still serves /health/*.
• Fail-closed — a route registered without a table entry answers 403 naming ROUTE_OPERATIONS.
• /permissions/show — clean, persisted, effective-with-flags, inert patterns, reachable under total deny, still 401 without the token.
• Verb/risk agreement — the three structural checks above.

Mutation-checked: commenting out the app-level dependency fails exactly 5 of them, so they are load-bearing rather than decorative.

A trap worth flagging for review

FastAPI 0.137 stopped flattening include_router eagerly — app.routes holds 35 lazy _IncludedRouter proxies instead of the 236 routes they stand for, and nothing materialises them (not app.openapi(), not TestClient startup). Request handling is unaffected, but a completeness test that walked app.routes naively would audit four routes, find nothing wrong, and pass. _iter_api_routes recurses through original_router; the docstring says why, because this is exactly the false-pass shape a coverage test must not have.

Merge-order notes

• serve: honor the root-level --config-dir, and fix two stale permissions comments #681 (open) also edits src/keboola_agent_cli/permissions.py. Both PRs rewrite the same three-line http.* comment (it fixes the false claim; this makes the claim true). Expect a small conflict there — resolve in favour of whichever lands second, keeping this PR's wording if that is this one.
• serve: missing mirrors — describe-batch (inline payload), unload-table; job run lacks idempotency_key #657, if implemented, adds REST routes and will need matching ROUTE_OPERATIONS entries. The completeness test names each missing route, so the failure is self-explaining rather than mysterious.

No version bump, no changelog entry (per CONTRIBUTING: those belong to the release PR). New behaviour is gated with the literal (since vNEXT) placeholder on every doc surface; context.py carries no version tag, matching the precedent set in #681.

Doc surfaces updated (convention #17)

docs/web-server.md (new "The session firewall applies to every route" section, plus two stale paragraphs that asserted the gap), docs/web-server-endpoints.md (regenerated), CLAUDE.md, plugins/kbagent/skills/kbagent/references/gotchas.md (the "A deny policy does NOT firewall the whole REST surface" entry was live and is now inverted, with the 0.90.1-and-older behaviour kept for readers on those versions), commands-reference.md, and commands/context.py's AGENT_CONTEXT. No CLI command added, renamed, or removed, so keboola-expert.md and SKILL.md need no change.

Update 2026-10-10

• Merged main (0.98.0). make check passes.
• Routes that main added after this PR are in the table: 4 routes from 0.96.x and the 7 semantic-layer scope routes from 0.97.0. The completeness test passes, so no live route is unclassified.
• The firewall dependency runs before the project-ID translation of serve (CLI-22). A denied request does not reach the project lookup.
• --scope organization writes are destructive-class over REST too. On the CLI, FLAG_ESCALATIONS makes an organization scope destructive-class, because the elevation is one-way. The route table checks only the base key, so before this change a REST caller under --deny-destructive could still elevate an item. Now PUT /semantic-layer/scope/{context_id}, POST /semantic-layer/models, POST /semantic-layer/items/{kind}, import, promote and build (with a model) check the escalated key when the request asks for organization scope. An inherited organization scope is checked the same way as on the CLI: only when a policy is active, through the scope of the target model. The item check uses the per-kind key, semantic-layer.add.<kind> --scope organization.
• The two scope DELETE routes (remove target projects, withdraw an elevation request) are write, as on the CLI. The test that every DELETE is destructive or admin lists them as exceptions.
• New tests: TestOrganizationScopeEscalation (typed and inherited organization scope, per-kind keys, no lookup without a policy). Mutation check: with the escalated check removed, 18 of the 20 new tests fail.
• Docs: docs/web-server.md, CLAUDE.md, gotchas.md and commands-reference.md say that organization-scope writes are destructive-class over REST.
• The granularity caveat above is gone. The item routes (POST /semantic-layer/items/{kind}, PUT and DELETE /semantic-layer/items/{kind}/{name}) now check the parent key and the leaf key semantic-layer.<verb>.<kind>, like the CLI. A policy that denies an exact leaf key or a glob such as semantic-layer.add.* now holds over REST too.
• Only GET and HEAD of a bootstrap path are exempt. The server reads the policy once, when it starts. Restart it after permissions set or reset.

[keboola-pr-reviewer-bot]

Verdict: needs_human (risk 3/5) · profile _default

Security-hardening change extends the serve permission firewall to every REST route; sound and well-tested, but auth-decision logic on a production service warrants human sign-off.

Suggested reviewers: keboola/keboola-cli-maintainers

[devin-ai-integration]

Devin Review found 7 potential issues.

[devin-ai-integration]

🟨 Updated firewall rules stay inactive

After the host tightens its persisted policy, enforce_route_permission keeps using the startup engine. REST calls remain allowed under the old policy until the server restarts.

(Refers to this code)

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🟡 Read-only scaffolds blocked by write policy

Under a write-denying policy, config.new blocks component scaffolding. generate_scaffold only fetches metadata and returns generated files; read-only users lose this endpoint.

Learn more

The component scaffold endpoint builds an in-memory scaffold and returns its generated files. It never calls the Storage configuration creation API. The CLI's config new has a separate optional --push path, but this REST route does not expose it. Mapping this route to the write-class config.new key prevents it from running under --deny-writes even though it is a read-only request.

Example: A server started with a policy denying cli:write accepts component detail requests but answers 403 to POST /components/my-component/scaffold. Without the policy, that request only returns the proposed files.

Recommended fix: Add a distinct read-class registry key for the scaffold-only operation, mark it serve-only if it has no matching CLI leaf, and map the scaffold route to it. Check both exact-name policies and category policies in a regression test.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🟡 Sharing edges blocked by unrelated policy

With only sharing.edges allowed, lineage.show blocks the cross-project edges endpoint. The edges handler serves the same sharing graph, so matching policies produce different results.

Suggested change

	("GET", "/lineage/edges"): "lineage.show",
	("GET", "/lineage/edges"): "sharing.edges",

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🔍 Inline guard recognition has a narrow scope

_declares_inline_permission inspects direct route dependencies only. A future router-level or nested require_permission declaration can be refused as unclassified despite having a guard.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🔍 Inherited-scope tests bypass model resolution

The escalation tests mock child_scope, so they do not cover real model lookup and inheritance. Consider one service-backed request under an active policy.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🟥 Read-only permission allows lineage writes

With cli:write denied, lineage.build still allows POST /lineage/build. The build handler writes files and can run sync.pull_all when refresh is true.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🟥 Read-only policy permits password resets

Under cli:write denial, workspace.password still permits a workspace password reset. The password handler rotates credentials rather than merely reading them.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[soustruh]

The firewall design holds up. I found no blocking issue. Two small findings are inline: one stale version number in the docs, one note on the exempt-path check.

Checked:

• Fail-closed: the app-level dependency runs first, an unmapped route answers 403, and inline require_permission routes (/auth/*, /merge-requests/*) are skipped correctly. The completeness test walks the lazy _IncludedRouter proxies and is cross-checked against the OpenAPI paths, so it cannot pass by seeing no routes.
• Classification: I compared every ROUTE_OPERATIONS value with its OPERATION_REGISTRY class. Every non-GET route maps to write or higher, except the eight documented POST-as-read routes plus workspace.password, which is also read on the CLI. Every DELETE is destructive or admin, except the two scope DELETEs, which are write as on the CLI. Both exceptions are documented.
• Scope escalation: typed and inherited organization scope match resolve_scope_targets and gate_inherited_organization_scope (same keys, same model lookup only when a policy is active). All five item kinds, models, scope PUT, import, promote and build are covered. Edit routes have no scope field.
• GET /permissions/show is read-only, returns the effective policy, and cannot widen it. The firewall runs before project-ID translation. The new tests pass locally.

[keboola-pr-reviewer-bot]

Verdict: needs_human (risk 4/5) · profile keboola-mcp-server

Authorization control-flow change extending the permission firewall to every kbagent serve REST route — needs a human despite high quality.

[zajca]

Actionable findings from the automated review.