feat(semantic-layer): metastore scope/ACL support (targeted + org-wide) [AI-3790]

CLAUDE.md

@@ -1054,7 +1054,7 @@ kbagent dev-portal deprecate --app VENDOR.APP_ID [--identity A] [--dry-run]
 kbagent encrypt values --project ALIAS --component-id ID --input JSON|@file|- [--output-file PATH]
 
 kbagent semantic-layer model list --project P
-kbagent semantic-layer model create --project P --name N [--description D] [--sql-dialect Snowflake]
+kbagent semantic-layer model create --project P --name N [--description D] [--sql-dialect Snowflake] [--scope project|organization|targeted] [--target-project ALIAS|ID ...]
 kbagent semantic-layer model delete --project P --model M [--yes]
 kbagent semantic-layer show --project P [--model M] [--type dataset|metric|relationship|constraint|glossary]
 kbagent semantic-layer search-context --project P [--pattern G ...] [--type model|dataset|metric|relationship|constraint|glossary|all] [--limit N]
@@ -1092,19 +1092,55 @@ kbagent semantic-layer import --project P --file PATH [--model M] [--types T,T,.
 kbagent semantic-layer promote --from-project A --to-project B [--from-model M] [--to-model M] [--types T,T,...] [--dry-run] [--yes]
 kbagent semantic-layer build --project P [--model M] --tables T,T,... [--name N] [--dry-run] [--keep-on-failure] [--output PATH]
 kbagent semantic-layer token --encrypt --project P --component-id C
+# scope (PSGO-140, new): visibility scope for a semantic-layer item -- "project"
+#   (owner only), "organization" (every project in the org), or "targeted" (owner +
+#   explicit --target-project grants). --scope/--target-project are also accepted by `model
+#   create` and every `add <kind>` above. --scope omitted: `model create` makes a "project"
+#   item, `add <kind>` INHERITS its model's scope (an org-level model gets org-level children;
+#   a targeted model, its target projects); an inherited organization scope is permission-gated
+#   like a typed one, and a non-org-admin gets a 403 on it (pass --scope project). --target-project takes a registered alias OR a
+#   numeric project ID (repeatable or comma-separated; an alias must be on the owner's stack);
+#   without --scope targeted it exits 2. With --scope targeted and no --target-project: a real
+#   terminal launches an interactive picker over the other projects on the stack; --json fails
+#   fast (exit 2). A bad alias/ID/--scope/--type is a usage error (exit 2), never a traceback.
+#   Creating directly at --scope organization requires the organization-admin ROLE (403
+#   otherwise) -- an ordinary token instead uses `scope request-create` + an org-admin's
+#   `scope set --scope organization`. Elevation is ONE-WAY (no downgrade endpoint), which is why
+#   every `--scope organization` (here and on `model create` / `add <kind>`) is destructive-class
+#   (FLAG_ESCALATIONS). `edit`, `import --overwrite` and `promote` update in place (PUT): an item
+#   keeps its id, scope, grants and pending elevation request. `scope add|remove` merge into the
+#   grants client-side (not atomic; refused from a project that does not own the item, because
+#   the server hides the grants from a non-owner -- use `scope set --target-project` there).
+#   Verbs follow the CLI spec (#791): get / add / remove / set / request-create / request-delete /
+#   request-list; `scope set` takes exactly ONE of --scope organization, --target-project (replace
+#   the whole list) or --clear.
+kbagent semantic-layer scope get --project P --type model|dataset|metric|relationship|constraint|glossary --context-id ID
+kbagent semantic-layer scope add --project P --type T --context-id ID --target-project ALIAS|ID [--target-project ...]
+kbagent semantic-layer scope remove --project P --type T --context-id ID --target-project ALIAS|ID [--target-project ...]
+kbagent semantic-layer scope set --project P --type T --context-id ID (--scope organization | --target-project ALIAS|ID [...] | --clear) [--dry-run] [--yes]
+kbagent semantic-layer scope request-create --project P --type T --context-id ID
+kbagent semantic-layer scope request-delete --project P --type T --context-id ID
+kbagent semantic-layer scope request-list --project P --type T [--limit N] [--offset N]
+# scope over `kbagent serve`: GET/PUT /semantic-layer/scope/{context_id}, POST/DELETE
+#   .../target-projects, PUT/DELETE .../elevation-request, GET /semantic-layer/scope/elevation-requests;
+#   POST /semantic-layer/models and /items/{kind} take `scope` + `target_projects`.
 kbagent semantic-layer reference-data list --project P [--model M]
 kbagent semantic-layer reference-data get --project P (--id ID | --dimension D)
 kbagent semantic-layer reference-data set --project P [--model M] --dimension D --members-file PATH [--dataset-id T] [--description X]
 kbagent semantic-layer reference-data delete --project P --id ID [--yes]
 # Alias: `kbagent sl ...` (hidden) is equivalent to `kbagent semantic-layer ...`.
-# semantic-layer REQUIRES A MASTER (project admin) Storage token (#711): the Metastore's
-#   auth gate -- unlike the Storage API -- rejects every valid non-master token with an opaque
-#   401 "Failed to create project scope" (the underlying MasterTokenRequiredError is swallowed
-#   server-side). kbagent reclassifies exactly that 401 to MISSING_MASTER_TOKEN (exit 3) with
-#   the remedy; other unexplained 401s anywhere map to AUTH_REJECTED instead of the false
-#   "Invalid or expired token" (INVALID_TOKEN stays for 401s that DO blame the credential).
-#   Version gate for this entry lives in gotchas.md -- `(since vNEXT)` cannot be written on
-#   these `# ` comment lines (check_version_gates.py parses them as ATX headings).
+# semantic-layer READS work with any valid, non-disabled, non-expired Storage token
+#   (vNEXT, PSGO-282): the Metastore no longer requires a master token for GET/List. WRITES
+#   (add/edit/remove/import/promote/build/scope add|remove|set) still need a project-admin token
+#   (master token or any admin-role user token) -- a non-admin token 403s on the write. Before
+#   PSGO-282 the Metastore's auth gate rejected EVERY valid non-master token, including reads,
+#   with an opaque 401 "Failed to create project scope" (#711, MasterTokenRequiredError swallowed
+#   server-side); kbagent still reclassifies exactly that 401 to MISSING_MASTER_TOKEN (exit 3) as
+#   a safety net for a deployment that predates the fix. Other unexplained 401s anywhere map to
+#   AUTH_REJECTED instead of the false "Invalid or expired token" (INVALID_TOKEN stays for 401s
+#   that DO blame the credential). Version gate for this entry lives in gotchas.md -- `(since
+#   vNEXT)` cannot be written on these `# ` comment lines (check_version_gates.py parses them as
+#   ATX headings).
 
 kbagent http get PATH [--timeout SECONDS]
 kbagent http post PATH [--body JSON|@file|-] [--timeout SECONDS]

docs/error-codes.md

@@ -19,7 +19,7 @@ of `ErrorCode` in `src/keboola_agent_cli/errors.py`.
 | `INVALID_TOKEN` | Storage API token is invalid or expired |
 | `ACCESS_DENIED` | Token lacks the required permission for this API call |
 | `PERMISSION_DENIED` | Operation blocked by the active kbagent permission policy |
-| `MISSING_MASTER_TOKEN` | Operation requires a master (admin) Storage token (e.g. `token create`, `config oauth-url` pre-flight, and every `semantic-layer` command -- the Metastore's opaque 401 `Failed to create project scope` is its master-token gate and is reclassified to this code); maps to exit 3 |
+| `MISSING_MASTER_TOKEN` | Operation requires a master (admin) Storage token (e.g. `token create`, `config oauth-url` pre-flight, and a `semantic-layer` WRITE against a non-admin token -- `semantic-layer` reads work with any valid token since PSGO-282; the Metastore's opaque 401 `Failed to create project scope` was its master-token gate pre-PSGO-282 and is reclassified to this code as a safety net); maps to exit 3 |
 | `UNAUTHORIZED` | `kbagent serve` rejected the request's Bearer token |
 | `AUTH_REJECTED` | Upstream answered HTTP 401 but its own error text did not blame the credential. Same exit 3 as `INVALID_TOKEN`; rotating the token is not the fix. (The Metastore's `Failed to create project scope` 401 has a known cause and lands on `MISSING_MASTER_TOKEN` instead) |
 

docs/web-server-endpoints.md

@@ -9,7 +9,7 @@ auth, and the concepts behind these routes live in
 [`web-server.md`](web-server.md); a running server serves the same spec
 interactively at `/docs` (Swagger) and `/openapi.json`.
 
-**248 operations** across **216 paths** and **31 routers**.
+**255 operations** across **220 paths** and **31 routers**.
 
 Paths are shown as the server registers them. Reaching them through the
 Node BFF (or single-process `--ui` mode) prefixes every path with `/api`.
@@ -388,7 +388,7 @@ Build and query cross-project data lineage (table-level and column-level). Mirro
 | `GET` | `/lineage/walk` | Walk lineage graph from a node |
 | `GET` | `/lineage/mermaid` | Render lineage as Mermaid |
 
-### `semantic-layer` (21 operations)
+### `semantic-layer` (28 operations)
 
 Model, validate, import/export, diff, promote, and build semantic layer artifacts (datasets, metrics, relationships, constraints, glossary). Mirrors `kbagent semantic-layer *`.
 
@@ -415,6 +415,13 @@ Model, validate, import/export, diff, promote, and build semantic layer artifact
 | `PUT` | `/semantic-layer/reference-data` | Create or replace a reference-data record |
 | `GET` | `/semantic-layer/reference-data/{record_id}` | Get one reference-data record |
 | `DELETE` | `/semantic-layer/reference-data/{record_id}` | Delete a reference-data record |
+| `GET` | `/semantic-layer/scope/elevation-requests` | List scope-elevation requests |
+| `GET` | `/semantic-layer/scope/{context_id}` | Get an item's scope |
+| `PUT` | `/semantic-layer/scope/{context_id}` | Set an item's scope or target projects |
+| `POST` | `/semantic-layer/scope/{context_id}/target-projects` | Add target projects |
+| `DELETE` | `/semantic-layer/scope/{context_id}/target-projects` | Remove target projects |
+| `PUT` | `/semantic-layer/scope/{context_id}/elevation-request` | Request scope elevation |
+| `DELETE` | `/semantic-layer/scope/{context_id}/elevation-request` | Withdraw scope elevation request |
 
 ## AI & Tools
 

plugins/kbagent/agents/keboola-expert.md

@@ -151,8 +151,9 @@ been retired, so its absence is NOT a promise (see §1 Rule 6).
 | Call the running `kbagent serve` from a scheduled-agent subprocess | `kbagent http get/post/patch/delete <PATH>` -- uses the `KBAGENT_SERVE_URL` + `KBAGENT_SERVE_TOKEN` the scheduler injects; `http get /openapi.json` discovers endpoints | forking `kbagent <command>` (also fine -- `KBAGENT_CONFIG_DIR` is propagated) | `curl $KBAGENT_SERVE_URL/...` by hand (loses the auth header, error mapping and JSON formatting `kbagent http` adds) |
 | Launch the web UI for an end-user | `kbagent serve --ui [--port PORT]` -- one FastAPI process mounts the bundled React SPA and sets an HttpOnly `kbagent_session` cookie, so SSE works with no token in the URL or JS heap | the legacy Vite + Node BFF dev flow (`web/README.md`) | inventing a `--token-in-url` flag; running uvicorn directly against the dist (the cookie bootstrap only fires from `serve --ui`) |
 | Schedule / manage Agent Tasks | `kbagent agent <verb>` -- CRUD, `run [--stream]`, history, `test`/`cron-preview`/`prompt-improve`. Local-only; cron firing needs `kbagent serve`. See [agent-tasks-cli-workflow](../skills/kbagent/references/agent-tasks-cli-workflow.md) | `kbagent http <verb> /agents...` inside scheduled subprocesses | hand-editing `agents.json` |
-| Read a semantic-layer model (models, metrics, datasets, constraints) | `kbagent --json semantic-layer show --project P [--model M] [--type metric\|dataset\|relationship\|constraint\|glossary]`; `model list`; `search-context` / `get-context` for glob/id lookup; `validate [--deep]` before trusting one. The WHOLE `semantic-layer` family needs a MASTER token: a valid non-master token gets `MISSING_MASTER_TOKEN` (0.92.0+, #711; the Metastore's opaque 401 "Failed to create project scope") -- register a master token, do not escalate | -- | hand-rolled `httpx` loops against `metastore.*.keboola.com` (bypasses retry/backoff and the kbagent error envelope) |
-| ANY semantic-layer write (add / edit / remove / import / promote / build) | `kbagent semantic-layer export` FIRST (the metastore has no soft-delete and no version history -- the snapshot is the only restore path), then the write, `--dry-run` where offered. `Read` [semantic-layer-workflow.md](../skills/kbagent/references/semantic-layer-workflow.md) before starting: it carries the per-verb recipes, the rename cascade and the promote classification | `semantic-layer diff` (`--project-a/-b` or `--file-a/-b`) to confirm what a write would change | raw metastore REST (no rollback, no orphan scan, no modelUUID rewrite); a write with no export taken |
+| Read a semantic-layer model (models, metrics, datasets, constraints) | `kbagent --json semantic-layer show --project P [--model M] [--type metric\|dataset\|relationship\|constraint\|glossary]`; `model list`; `search-context` / `get-context` for glob/id lookup; `validate [--deep]` before trusting one. Any valid, non-disabled, non-expired Storage token works for reads (vNEXT, PSGO-282 -- the metastore no longer requires a master token for GET/List); a stale `MISSING_MASTER_TOKEN` on a plain read means an outdated kbagent still carrying the old reclassification, not a real gate | -- | hand-rolled `httpx` loops against `metastore.*.keboola.com` (bypasses retry/backoff and the kbagent error envelope) |
+| ANY semantic-layer write (add / edit / remove / import / promote / build / scope add|remove|set|request-create) | `kbagent semantic-layer export` FIRST (the metastore has no soft-delete and no version history -- the snapshot is the only restore path), then the write, `--dry-run` where offered. Needs a token whose Storage Admin role is `admin` on the project (a master token qualifies, but so does any project-admin user token as of PSGO-282) -- a non-admin token still gets `MISSING_MASTER_TOKEN`/403 on writes, just not on reads. `Read` [semantic-layer-workflow.md](../skills/kbagent/references/semantic-layer-workflow.md) before starting: it carries the per-verb recipes, the rename cascade and the promote classification | `semantic-layer diff` (`--project-a/-b` or `--file-a/-b`) to confirm what a write would change | raw metastore REST (no rollback, no orphan scan, no modelUUID rewrite); a write with no export taken |
+| Share a semantic-layer object across projects, or make it org-wide | **ASK THE USER which project(s) before EVER passing `--scope organization\|targeted` (on `model create`/`add <kind>`) or running `scope set --scope organization`/`scope request-create` -- never guess or default to widening visibility.** Once the user names the target(s): `--scope targeted --target-project ALIAS\|ID` grants specific projects; `scope add\|remove --project P --type T --context-id ID --target-project ALIAS\|ID` adjusts an existing item's grants from the owning project (a merge), `scope set --target-project ...` / `--clear` replaces the list; org-wide needs `scope request-create` then an org-admin's `scope set --scope organization --yes` (preview with `--dry-run`) -- **irreversible, no downgrade endpoint, destructive-class**. `add <kind>` without `--scope` inherits its model's scope. `scope get --type T --context-id ID` shows the current state first. See [metastore-scope-workflow.md](../skills/kbagent/references/metastore-scope-workflow.md) | `scope request-list --type T` to find items an org-admin should act on | passing `--scope organization\|targeted` or running `scope set --scope organization` without the user having named the target project(s) first; assuming `--scope organization` succeeds on a non-admin token (it 403s -- creating directly at organization scope needs the SAME organization-admin role as elevating) |
 | User asks to "log in" / authenticate via browser / register a session's projects | ATTENDED session + a background shell: run `kbagent auth login --device-code --stack URL --register-projects` in a **BACKGROUND** shell (capture stdout+stderr, human mode -- `--json` puts the panel on stderr), relay the verification URL + user code to the user, then poll `kbagent --json auth status` (exit 0 = signed in, exit 3 = not yet). The human's part is approving in the browser, not typing the command. No background shell -> hand the plain `auth login --stack URL --register-projects` to the user's terminal. To register projects from an EXISTING session, `kbagent auth register-projects --all` or `--project-id ID` is non-interactive and agent-safe | -- | running `auth login` in a FOREGROUND tool shell (~120 s timeout kills it mid-flight); running it unattended (nobody can approve); re-running it blind without checking `auth status` first (orphans a session); the flagless `register-projects` picker unattended; reading the token out of `auth.json`; using a numeric project id as an alias |
 | User has NO Keboola account / project at all and asks to get started | `kbagent project create --url URL [--project ALIAS] [--name NAME] [--backend snowflake\|bigquery]` (0.95.0+) -- provisions a real project, stores its session and registers the alias in one call, agent-runnable. **Then relay the result's `confirm_url` to the human verbatim and say the project is owned by nobody until they open it**; after they confirm, the agent session is revoked by design -> `kbagent auth login --stack URL` | the user creating the project in the Keboola UI, then `auth login` / `project add --token` | calling it on a stack where a session already exists (exit 5 -- one session per stack); reporting success without the confirm link (an unclaimed project is a billable orphan); retrying it after a 5xx/429/503 without being asked (each success creates a new organization + project + credit grant) |
 | CI task has account credentials | `kbagent auth login-password --email E (--password-stdin \| --password P) [--totp-secret SEED]` (0.84.0+), agent-runnable | a static Storage token | `auth login` unattended |
@@ -250,11 +251,13 @@ its absence is NOT a promise the entry is version-independent (see §1 Rule 6).
 **A 401 is not always `INVALID_TOKEN` (0.92.0+)**
 - A 401 whose body names something other than a bad credential now raises
   `AUTH_REJECTED`; the semantic-layer family's opaque metastore 401
-  reclassifies to `MISSING_MASTER_TOKEN` instead (see its matrix row).
-  Exit code is unchanged (3) for all three -- code checking
-  `error.code == "INVALID_TOKEN"` alone now misses two real cases; accept
-  all three before concluding the credential itself is bad. 401/403/404
-  now also carry `[exceptionId: ...]` -- quote it when escalating. gotchas.md.
+  reclassified to `MISSING_MASTER_TOKEN` on a write against a non-admin
+  token (see its matrix row) -- since PSGO-282 (vNEXT) a plain read never
+  hits this at all, any valid token passes. Exit code is unchanged (3) for
+  all three -- code checking `error.code == "INVALID_TOKEN"` alone still
+  misses two real cases; accept all three before concluding the credential
+  itself is bad. 401/403/404 now also carry `[exceptionId: ...]` -- quote it
+  when escalating. gotchas.md.
 
 **Finding an existing Storage token**
 - `kbagent token list -p P` (0.86.0+) -- the only source of the `--token-id`