Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .agents/skills/test-release-canary/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,7 @@ Loopback registration auto-derives the gateway name to `openshell` if `--name` i
| `macos`/`ubuntu-deb`/`fedora` job fails on `install.sh` | Dev release missing an asset, checksum mismatch, or `install.sh` regression on this branch. | Job log around the `curl … install.sh \| sh` step. |
| Sandbox create or exec fails | Published sandbox and supervisor artifacts are missing, incompatible, or cannot establish the protected runtime channel. | Gateway logs plus Docker, Podman, VM, Snap, or Kubernetes runtime diagnostics for the job. |
| `macos`/`ubuntu-deb`/`fedora` job fails on `openshell status` | Local gateway service did not start (systemd/brew/podman). Often a driver issue. | Service logs in the job log; `OPENSHELL_COMPUTE_DRIVER` env in the "Ensure …" step. |
| `ubuntu-snap-system-docker` fails during `install.sh` | System Docker was unavailable, the edge revision or automatic interfaces were unavailable, or the gateway did not become reachable. | Failure diagnostics dump system Docker, snap service/connection/change state, gateway and snapd journals, snap logs, and port 17670 listeners. |
| `ubuntu-snap-system-docker` fails during `install.sh` | System Docker was unavailable to the runner user, the edge revision or automatic interfaces were unavailable, or the user gateway did not become reachable. | Failure diagnostics dump system Docker, snap service/connection/change state, user and legacy gateway journals, snap logs, and port 17670 listeners. |
| `ubuntu-snap-system-docker` fails during the prover checks | The prover artifact is missing or packaged for the wrong architecture, `openshell.prover` is not exposed or confined to read the test policies, or its solver linkage is not runnable. | The `Verify Snap installation` and `Check a policy boundary with the Snap prover` steps, plus `snap info openshell` and `snap connections openshell`. |
| `ubuntu-snap-docker-preflight` unexpectedly succeeds | The installer no longer fails before installing the OpenShell snap when Docker is absent or supplied by the Docker snap. | Inspect `install.log`, `docker-snap.log`, `snap list`, and snapd changes. |
| `kubernetes` job fails on `helm install --wait` | Chart did not deploy in 5 min — usually image pull failure or readiness probe failing. | "Diagnostics on failure" step dumps `helm status`, manifest, pod describe, pod logs. |
Expand Down
21 changes: 14 additions & 7 deletions .github/workflows/release-canary.yml
Original file line number Diff line number Diff line change
Expand Up @@ -221,9 +221,10 @@ jobs:
- name: Install and check status
run: |
set -euo pipefail
sudo systemctl set-environment \
systemctl --user set-environment \
"OPENSHELL_TELEMETRY_ENABLED=${OPENSHELL_TELEMETRY_ENABLED}"
curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/${{ github.event.workflow_run.head_sha || github.sha }}/install.sh | sh
curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/${{ github.event.workflow_run.head_sha || github.sha }}/install.sh |
DOCKER_HOST=unix:///tmp/nonexistent-docker.sock DOCKER_CONTEXT=nonexistent sh
sudo snap list openshell
if sudo snap list docker >/dev/null 2>&1; then
echo "install.sh unexpectedly installed the Docker snap" >&2
Expand All @@ -234,8 +235,12 @@ jobs:
sudo snap connections openshell | grep -E '^docker +openshell:docker +:docker +'
openshell --version
openshell.prover --version
sudo snap services openshell
sudo journalctl -b -u snap.openshell.gateway.service --no-pager |
snap services openshell
systemctl --user is-enabled --quiet snap.openshell.user-gateway.service
systemctl --user is-active snap.openshell.user-gateway.service
! systemctl is-enabled --quiet snap.openshell.system-gateway.service
! systemctl is-active --quiet snap.openshell.system-gateway.service
journalctl --user -u snap.openshell.user-gateway.service --no-pager |
grep -F "mTLS user authentication enabled"
openshell gateway list | grep -F "https://127.0.0.1:17670"
openshell status
Expand Down Expand Up @@ -280,10 +285,12 @@ jobs:
sudo snap services openshell
sudo snap connections openshell
sudo snap changes
sudo systemctl status snap.openshell.gateway.service --no-pager
sudo journalctl -b -u snap.openshell.gateway.service --no-pager -n 300
systemctl --user status snap.openshell.user-gateway.service --no-pager
journalctl --user -u snap.openshell.user-gateway.service --no-pager -n 300
sudo systemctl status snap.openshell.system-gateway.service --no-pager
sudo journalctl -b -u snap.openshell.system-gateway.service --no-pager -n 300
sudo journalctl -b -u snapd.service --no-pager -n 300
sudo snap logs openshell.gateway -n=300
snap logs openshell.user-gateway -n=300
sudo ss -ltnp '( sport = :17670 )'

ubuntu-snap-docker-preflight:
Expand Down
2 changes: 1 addition & 1 deletion CI.md
Original file line number Diff line number Diff line change
Expand Up @@ -543,7 +543,7 @@ These workflows run after merge to publish dev/tagged artifacts and verify them.
| `.github/workflows/release-tag.yml` | Publishes tagged stable releases and manually dispatched pre-releases. Its automatic tag trigger excludes `-pre.*`. Both require the currently implemented qualification profile to pass before publication; the summary identifies the remaining RFC 0014 coverage. Failed candidates retain build artifacts and evidence in Actions storage. Source-SHA OCI images remain available as qualification inputs. |
| `.github/workflows/snap-package.yml` | Builds Snap and component artifacts for Release Dev and Release Tag without Store credentials or publication. |
| `.github/workflows/snap-publish.yml` | Uploads existing Snap and component artifacts to the Store without rebuilding. Release Dev calls it directly for `latest/edge` after Snap builds and requires it to succeed before creating the dev release. Release Tag calls it for `latest/stable` only for stable releases, after qualification and release assembly succeed. |
| `.github/workflows/release-canary.yml` | Smoke-tests published dev artifacts in the `macos`, `ubuntu-deb`, `ubuntu-snap-system-docker`, `fedora`, and `kubernetes` (kind + Helm) jobs. Each job reaches its gateway and creates, exercises, and deletes a sandbox. The Snap lanes verify a compatible system Docker lifecycle and `ubuntu-snap-docker-preflight` tests fail-fast behavior when Docker is absent or supplied by the Docker snap. The positive Snap lane also runs a local policy containment check with the packaged prover. It runs automatically after `Release Dev` succeeds and supports manual dispatch (`gh workflow run release-canary.yml --ref <branch>`). See the `test-release-canary` skill for the playbook and local kind reproduction. |
| `.github/workflows/release-canary.yml` | Smoke-tests published dev artifacts in the `macos`, `ubuntu-deb`, `ubuntu-snap-system-docker`, `fedora`, and `kubernetes` (kind + Helm) jobs. Each job reaches its gateway and creates, exercises, and deletes a sandbox. The `ubuntu-snap-system-docker` job verifies a compatible system Docker lifecycle, while `ubuntu-snap-docker-preflight` tests fail-fast behavior when Docker is absent or supplied by the Docker snap. The positive Snap lane also runs a local policy containment check with the packaged prover. It runs automatically after `Release Dev` succeeds and supports manual dispatch (`gh workflow run release-canary.yml --ref <branch>`). See the `test-release-canary` skill for the playbook and local kind reproduction. |

## Required status contexts

Expand Down
4 changes: 3 additions & 1 deletion deploy/man/openshell-gateway.8.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,9 @@ The Debian and Ubuntu systemd user unit runs preflight before certificate
generation, while retaining its EnvironmentFile and bare ExecStart behavior. The
Snap wrapper replays its effective daemon arguments through preflight. It first
uses a nonempty OPENSHELL_GATEWAY_CONFIG. Otherwise it passes the canonical
SNAP_COMMON/gateway.toml path whenever it exists or is a symlink. A broken symlink
service-specific gateway.toml path whenever it exists or is a symlink. The legacy
system service uses SNAP_COMMON/gateway.toml; the user service uses
SNAP_USER_COMMON/.config/openshell/gateway.toml. A broken symlink
fails preflight before the gateway is started. Correct or manually migrate an
operator-owned v1 file, then run preflight again before restarting the service.

Expand Down
89 changes: 81 additions & 8 deletions docs/about/installation.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -132,31 +132,104 @@ The snap installs the standalone policy prover as `openshell.prover`. The
The prover reads local policy files through the `home` interface and does not
connect to the gateway.

The gateway runs as a system service at `https://127.0.0.1:17670` and reads `/var/snap/openshell/common/gateway.toml`. It requires a client certificate. The install script copies that certificate to the installing user's Snap state and registers the gateway automatically. If you installed with `sudo snap install openshell`, give each trusted user the certificate and register the gateway from that user's account:
The gateway runs at `https://127.0.0.1:17670`. It reads `~/snap/openshell/common/.config/openshell/gateway.toml` when that file exists. Its database defaults to `~/snap/openshell/common/.local/state/openshell/gateway/openshell.db`, and its TLS material lives under `~/snap/openshell/common/.local/state/openshell/tls`. Register it from the same user account:

```shell
snap services openshell.user-gateway
openshell gateway add https://127.0.0.1:17670 --local --name openshell
openshell status
```

Existing installations upgraded from the former `openshell.gateway` service retain their state under `/var/snap/openshell/common` through `openshell.system-gateway`. Existing HTTPS registrations continue to work because the endpoint and TLS identity do not change.

If this user still has an old HTTP registration, or has not previously been enrolled, wait for the system gateway to be active, copy its client credentials into the user's Snap state, and register its HTTPS endpoint:

```shell
snap services openshell.system-gateway
d=~/snap/openshell/common/.local/state/openshell/tls
mkdir -p -m 700 "$d" "$d/client"
sudo install -o "$USER" -m 600 /var/snap/openshell/common/tls/ca.crt "$d/"
sudo install -o "$USER" -m 600 -t "$d/client" \
/var/snap/openshell/common/tls/client/tls.crt /var/snap/openshell/common/tls/client/tls.key
sudo install -o "$USER" -g "$(id -gn)" -m 600 \
/var/snap/openshell/common/tls/ca.crt "$d/"
sudo install -o "$USER" -g "$(id -gn)" -m 600 \
/var/snap/openshell/common/tls/client/tls.crt "$d/client/"
sudo install -o "$USER" -g "$(id -gn)" -m 600 \
/var/snap/openshell/common/tls/client/tls.key "$d/client/"
openshell gateway list
old_registration="NAME_FROM_GATEWAY_LIST"
openshell gateway remove "$old_registration"
openshell gateway add https://127.0.0.1:17670 --local --name openshell
openshell status
```

Keep the client key private.
Set `old_registration` to the name shown by `openshell gateway list`, and omit the variable assignment and `openshell gateway remove` when no registration exists. Older direct Snap instructions may have used `openshell-gateway`, while the install script uses `openshell`.

To install a locally built snap, connect its interfaces manually. The gateway may reach systemd's start limit before Docker is connected, so reset the failed unit and restart the gateway after connecting the interfaces:
### Change the Snap gateway mode

The user and system gateways have separate configuration, databases, and cryptographic identities. Changing modes does not migrate gateway or sandbox state and interrupts active sandbox sessions. Back up any state you need and delete active sandboxes before changing modes.

First disable both services and move the current user TLS directory to a timestamped backup. The directory can contain CA, server, client, and sandbox JWT private keys, so keep every `tls-backup-*` directory private:

```shell
sudo snap set openshell gateway-mode=disable
state="$HOME/snap/openshell/common/.local/state/openshell"
tls="$state/tls"
if test -e "$tls" || test -L "$tls"; then
backup="$state/tls-backup-$(date +%s%N)"
while test -e "$backup" || test -L "$backup"; do
backup="$state/tls-backup-$(date +%s%N)"
done
mv -- "$tls" "$backup"
printf 'Previous gateway TLS state saved to: %s\n' "$backup"
fi
```

To use the user gateway, start it and replace the previous CLI registration after it generates new credentials:

```shell
sudo snap set openshell gateway-mode=user
systemctl --user is-active snap.openshell.user-gateway.service
openshell gateway list
old_registration="NAME_FROM_GATEWAY_LIST"
openshell gateway remove "$old_registration"
openshell gateway add https://127.0.0.1:17670 --local --name openshell
openshell status
```

To use the system gateway, start it, wait for it to generate its root-owned credentials, then copy its client bundle and replace the previous CLI registration:

```shell
sudo snap set openshell gateway-mode=system
sudo snap services openshell.system-gateway
tls="$HOME/snap/openshell/common/.local/state/openshell/tls"
mkdir -p -m 700 "$tls" "$tls/client"
sudo install -o "$USER" -g "$(id -gn)" -m 600 \
/var/snap/openshell/common/tls/ca.crt "$tls/"
sudo install -o "$USER" -g "$(id -gn)" -m 600 \
/var/snap/openshell/common/tls/client/tls.crt "$tls/client/"
sudo install -o "$USER" -g "$(id -gn)" -m 600 \
/var/snap/openshell/common/tls/client/tls.key "$tls/client/"
openshell gateway list
old_registration="NAME_FROM_GATEWAY_LIST"
openshell gateway remove "$old_registration"
openshell gateway add https://127.0.0.1:17670 --local --name openshell
openshell status
```

Set `old_registration` to the name shown by `openshell gateway list`, and omit the variable assignment and remove command when no registration exists. If a transition is interrupted while both services are disabled, select `gateway-mode=user` or `gateway-mode=system` to resume it. Keep the TLS backup until the selected gateway works and can create a sandbox.

To install a locally built snap, connect its interfaces manually. The user gateway may reach systemd's start limit before Docker is connected, so reset the failed unit and restart it from the affected user session after connecting the interfaces:

```shell
sudo snap install ./openshell_*.snap --dangerous
sudo snap connect openshell:log-observe
sudo snap connect openshell:system-observe
sudo snap connect openshell:docker :docker
sudo systemctl reset-failed snap.openshell.gateway.service
sudo snap restart openshell.gateway
systemctl --user reset-failed snap.openshell.user-gateway.service
snap start --user openshell.user-gateway
```

For an installation upgraded from a legacy system gateway, use `sudo systemctl reset-failed snap.openshell.system-gateway.service` and `sudo snap restart openshell.system-gateway` instead.

## Kubernetes

Deploy the gateway to a cluster with the OpenShell Helm chart. See [Kubernetes Setup](/kubernetes/setup).
Expand Down
8 changes: 5 additions & 3 deletions docs/how-it-works/gateways/configuration.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ Package-managed gateways use either built-in defaults or a package-seeded TOML f
| Homebrew | `$XDG_CONFIG_HOME/openshell/gateway.toml` when it exists, otherwise the Homebrew prefix config such as `/opt/homebrew/var/openshell/gateway.toml`. |
| Debian/Ubuntu | `$XDG_CONFIG_HOME/openshell/gateway.toml`, usually `~/.config/openshell/gateway.toml` for the systemd user service. |
| Fedora/RHEL RPM | `$XDG_CONFIG_HOME/openshell/gateway.toml`, usually `~/.config/openshell/gateway.toml`; the systemd user service seeds this file from the packaged template on first start. |
| Snap | `$SNAP_COMMON/gateway.toml`, usually `/var/snap/openshell/common/gateway.toml`. |
| Snap | New installations use `$SNAP_USER_COMMON/.config/openshell/gateway.toml`, usually `~/snap/openshell/common/.config/openshell/gateway.toml`. Upgraded legacy installations retain `$SNAP_COMMON/gateway.toml`, usually `/var/snap/openshell/common/gateway.toml`. |

The Fedora/RHEL RPM template leaves `[openshell.gateway].bind_address` unset. The gateway therefore uses its built-in `127.0.0.1:17670` primary listener. Host-networked Podman supervisors use that same loopback endpoint, so the primary listener does not need a wildcard address. Set `bind_address` explicitly only when clients must reach the primary multiplexed API through another interface.

Expand Down Expand Up @@ -1325,8 +1325,10 @@ Debian and Ubuntu run preflight from the systemd user unit before local certific
generation. The unit still loads the `gateway.env` environment file and starts the
gateway with no configuration arguments. Snap replays the exact effective daemon
arguments through preflight. It gives a nonempty `OPENSHELL_GATEWAY_CONFIG`
precedence; otherwise it validates and passes its canonical
`SNAP_COMMON/gateway.toml` only when that path exists in the filesystem. A broken
precedence; otherwise it validates and passes its canonical service-specific
path only when that path exists in the filesystem. The legacy system service
uses `$SNAP_COMMON/gateway.toml`; the user service uses
`$SNAP_USER_COMMON/.config/openshell/gateway.toml`. A broken
symlink is therefore rejected instead of being treated as absent.

Package startup does not modify an operator-owned v1 file. Back it up, follow
Expand Down
Loading
Loading