Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
43a9f59
test(testkit): add the testkit workspace for end-to-end suites
matthewgrossman Oct 19, 2026
2da1b0e
ci(tests): format, lint and test testkit in Branch Checks
matthewgrossman Oct 19, 2026
bd422fa
docs(tests): describe how to write and run testkit tests
matthewgrossman Oct 19, 2026
95ffb11
fix(testkit): give each sandbox its own cleanup timeout
matthewgrossman Oct 19, 2026
0320c22
docs(testkit): show how to exclude tests by capability
matthewgrossman Oct 19, 2026
bce1e43
docs: list tests/testkit in the project structure tables
matthewgrossman Oct 19, 2026
6a890d6
refactor(testkit): drop test IDs and capabilities for now
matthewgrossman Oct 19, 2026
e917506
docs(testkit): describe what testkit provides today
matthewgrossman Oct 19, 2026
f1bc726
refactor(testkit): find the gateway the way the CLI does
matthewgrossman Oct 19, 2026
e24542a
fix(testkit): keep labels when creating a sandbox with a policy
matthewgrossman Oct 21, 2026
99f995f
fix(testkit): use the registered gateway's credentials for an endpoint
matthewgrossman Oct 21, 2026
d4364ec
docs(tests): list testkit's gateway variables in precedence order
matthewgrossman Oct 21, 2026
09972dc
fix(testkit): apply other attributes to the generated test
matthewgrossman Oct 21, 2026
3b3f293
fix(testkit): show usage for testkit-ready --help
matthewgrossman Oct 21, 2026
af674ad
test(testkit): cover cleanup after pass, failure, and panic
matthewgrossman Oct 21, 2026
c8d4223
fix(testkit): reject #[should_panic] on testkit tests
matthewgrossman Oct 21, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/branch-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -221,6 +221,7 @@ jobs:
cargo fmt --manifest-path examples/governance-interceptor/Cargo.toml --all -- --check
cargo fmt --manifest-path examples/supervisor-middleware-content-guard/Cargo.toml --all -- --check
cargo fmt --manifest-path examples/isolation-backend-v8/Cargo.toml --all -- --check
cargo fmt --manifest-path tests/testkit/Cargo.toml --all -- --check

cargo-lockfiles:
name: Cargo lockfiles
Expand Down Expand Up @@ -277,6 +278,7 @@ jobs:
cargo clippy --locked --manifest-path examples/governance-interceptor/Cargo.toml --all-targets -- -D warnings
cargo clippy --locked --manifest-path examples/supervisor-middleware-content-guard/Cargo.toml --all-targets -- -D warnings
cargo clippy --locked --manifest-path examples/isolation-backend-v8/Cargo.toml --all-targets -- -D warnings
cargo clippy --locked --manifest-path tests/testkit/Cargo.toml --all-targets -- -D warnings

rust-test:
name: Rust test (${{ matrix.system }})
Expand Down Expand Up @@ -325,6 +327,7 @@ jobs:
run: |
cargo nextest run --locked --profile ci --workspace --features openshell-server/test-support
cargo nextest run --locked --config-file .config/nextest.toml --profile ci --manifest-path examples/supervisor-middleware-content-guard/Cargo.toml
cargo nextest run --locked --config-file .config/nextest.toml --profile ci --manifest-path tests/testkit/Cargo.toml

rust-postgres-test:
name: Rust PostgreSQL tests
Expand Down
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ Do not rely on this file for a full inventory. The detailed public and contribut
| `python/openshell/` | Python SDK | Python bindings and CLI packaging |
| `sdk/typescript/` | TypeScript SDK | Native Connect client, curated sandbox API, and generated protobuf types |
| `proto/` | Protobuf definitions | gRPC service contracts |
| `tests/testkit/` | E2E testkit | Support library and `#[testkit::test]` attribute for SDK-driven E2E suites |
| `deploy/` | Docker, Helm, K8s | Dockerfiles, Helm chart, manifests |
| `docs/` | Published docs | MDX pages, navigation, and content assets |
| `fern/` | Docs site config | Fern site config, components, and theme assets |
Expand Down
1 change: 1 addition & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -344,6 +344,7 @@ These are the primary `mise` tasks for day-to-day development:
| `sdk/typescript/` | TypeScript SDK (Connect client and generated protobuf bindings) |
| `proto/` | Protocol buffer definitions and [public API conventions](proto/README.md) |
| `tasks/` | `mise` task definitions and build scripts |
| `tests/testkit/` | Support library for SDK-driven end-to-end suites |
| `deploy/` | Dockerfiles, Helm chart, Kubernetes manifests |
| `docs/` | Published Fern docs source, navigation, and content assets |
| `fern/` | Fern site config, components, and theme assets |
Expand Down
11 changes: 11 additions & 0 deletions TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ crates/*/tests/ # Rust integration tests
python/openshell/ # Python unit tests (*_test.py suffix)
e2e/python/ # Python E2E tests (test_*.py prefix)
e2e/rust/ # Rust CLI E2E tests
tests/testkit/ # Support library for SDK-driven E2E suites
```

## Rust Tests
Expand Down Expand Up @@ -129,6 +130,16 @@ requires mTLS.
runtime. It covers the current schema and the Podman in-tree versus external
driver configuration boundary.

### testkit (`tests/testkit/`)

`testkit` is the support library for SDK-driven end-to-end suites. Tests are
`#[testkit::test]` functions with a required doc comment. They find the
gateway the same way the CLI does, through `OPENSHELL_GATEWAY_ENDPOINT`,
`OPENSHELL_GATEWAY`, or the active gateway, and `cx.cli` runs the CLI named
by `OPENSHELL_BIN`. See
[tests/testkit/README.md](tests/testkit/README.md) for how to write and run
tests.

### Python E2E (`e2e/python/`)

`mise run e2e:python` builds `openshell/e2e-python:dev` from
Expand Down
3 changes: 3 additions & 0 deletions tasks/rust.toml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ run = [
"cargo clippy --manifest-path examples/governance-interceptor/Cargo.toml --all-targets -- -D warnings",
"cargo clippy --manifest-path examples/supervisor-middleware-content-guard/Cargo.toml --all-targets -- -D warnings",
"cargo clippy --manifest-path examples/isolation-backend-v8/Cargo.toml --all-targets -- -D warnings",
"cargo clippy --manifest-path tests/testkit/Cargo.toml --all-targets -- -D warnings",
]
run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 lint native"
hide = true
Expand All @@ -45,6 +46,7 @@ run = [
"cargo fmt --manifest-path examples/governance-interceptor/Cargo.toml --all",
"cargo fmt --manifest-path examples/supervisor-middleware-content-guard/Cargo.toml --all",
"cargo fmt --manifest-path examples/isolation-backend-v8/Cargo.toml --all",
"cargo fmt --manifest-path tests/testkit/Cargo.toml --all",
]
hide = true

Expand All @@ -56,6 +58,7 @@ run = [
"cargo fmt --manifest-path examples/governance-interceptor/Cargo.toml --all -- --check",
"cargo fmt --manifest-path examples/supervisor-middleware-content-guard/Cargo.toml --all -- --check",
"cargo fmt --manifest-path examples/isolation-backend-v8/Cargo.toml --all -- --check",
"cargo fmt --manifest-path tests/testkit/Cargo.toml --all -- --check",
]
hide = true

Expand Down
1 change: 1 addition & 0 deletions tasks/test.toml
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@ run = [
"cargo test --workspace --exclude openshell-server",
"cargo test -p openshell-server --features test-support",
"cargo nextest run --config-file .config/nextest.toml --manifest-path examples/supervisor-middleware-content-guard/Cargo.toml",
"cargo nextest run --config-file .config/nextest.toml --manifest-path tests/testkit/Cargo.toml",
]
run_windows = "powershell -NoProfile -ExecutionPolicy Bypass -File tasks/scripts/windows-msvc.ps1 test-precommit native"
hide = true
Expand Down
Loading
Loading