Skip to content

bug(prover): containment check returns Inconclusive(InvalidWitness) for real expansions with system libz3 4.8.12 #4437

Open

Description

@mattj-monad

User Story

We run OpenShell gateways for an internal agent fleet and build the workspace from source outside
the Nix shell, linking the distro libz3-dev as CONTRIBUTING.md documents. Running the unit suite
on a fork rebased onto v0.1.4-pre.2, endpoint_selector_single_star_crosses_path_separators
fails every time on our machines.

Problem Statement

The network containment check decodes the solver's satisfying model into a concrete request and
rejects any model that is not a canonical request. The solver's action domain does not restrict the
path's characters, so which model comes back depends on the Z3 version. Z3 4.8.12 returns the path
/v1/\0/ for a /v1/* selector. The decoder correctly refuses it (the runtime canonicalizer never
presents control bytes to policy), and the check ends Inconclusive(InvalidWitness) instead of
Exceeds. Z3 4.13.3 returns a printable path and the same check reports Exceeds.

Impact / Why This Matters

  • The documented install path (sudo apt install libz3-dev) yields Z3 4.8.12 on Ubuntu 22.04,
    Ubuntu 24.04 and Debian 12, so mise run test fails out of the box there. CI does not see it
    because Rust tests run in the Nix shell, which pins a newer Z3. (deploy/docker/Dockerfile.ci
    installs noble's 4.8.12, but the Rust test jobs do not use it.)
  • openshell-prover-cli built against that Z3 reports a real boundary expansion as inconclusive
    and loses the counterexample. This fails closed, so it is not a soundness issue, but it hides
    exactly the evidence the check exists to produce.
  • Workarounds are building with vendored-z3 (needs GCC 13+ for <format>, so not on Ubuntu
    22.04's default toolchain) or prebuilt-z3 (links against a newer glibc than 22.04 has).

Acceptance Criteria

  • cargo test -p openshell-prover passes when linked against libz3 4.8.12 and 4.13.3.
  • A real expansion is reported as Exceeds with a counterexample that replays against the runtime
    Rego, independent of the Z3 version.
  • No result changes from Exceeds/Inconclusive to Within as a side effect.

Reproduction Steps

docker run --rm -v "$PWD":/src:ro rust:1.95.0-bookworm bash -c '
  apt-get update -qq && apt-get install -y -qq libz3-dev pkg-config clang libclang-dev protobuf-compiler >/dev/null
  cp -a /src /work && cd /work && rm -f rust-toolchain.toml
  cargo test -p openshell-prover --test runtime_parity'

The same command with rust:1.95.0-trixie (libz3 4.13.3) passes.

Suggested UX (if applicable)

Not applicable: no configuration or CLI change.

Environment

  • OpenShell v0.1.4-pre.2 and main at the time of writing (prover unchanged between them)
  • libz3 4.8.12 (Ubuntu 22.04 4.8.12-1, Ubuntu 24.04 4.8.12-3.1build1, Debian 12 4.8.12-3.1) fails
  • libz3 4.13.3 (Debian 13 4.13.3-1) passes

Logs

test endpoint_selector_single_star_crosses_path_separators ... FAILED
runtime expansion must exceed the boundary: Inconclusive(ReasonEvidence { code: InvalidWitness, reason: "solver returned a model that could not be decoded" })

Instrumented, the model Z3 4.8.12 returns for the path is "/v1/\u{0}/". Restricting the path to
0x21..=0x7e makes it return "/v1/\"/", which the decoder also rejects (" is not a literal
pchar).

Activity

  1. Kevinbeltran123 commented on Oct 16, 2026

    @Kevinbeltran123

    I hit the same InvalidWitness from a different trigger that does not depend on the Z3 version, so a fix scoped to Z3 4.8.12's model choice may not cover it.

    Context. In a personal project, I check an agent's policies against a boundary with openshell-prover. While checking GitLab-style paths (group%2Fproject), I found that a rule with a % escape that exceeds the boundary returns inconclusive instead of the counterexample. None of my current rules hold a %, but one compiled from GitLab's API would, and then I could not tell which request exceeds.

    Trigger. A candidate REST rule whose path holds a percent-escape (%2F, %41, or a bare %) and exceeds the boundary returns inconclusive (reason_code: invalid_witness, solver returned a model that could not be decoded) in under 0.1 s. The same rule without the escape returns exceeds_boundary naming its path, and %25 also works (path=/a%25b). allow_encoded_slash is not involved.

    Environment. openshell-prover 0.1.4 (Homebrew, macOS 26.6.2) linked against z3 5.1.0, so this happens on a current Z3, not only 4.8.12. Here the witness path is the rule's own literal path, so the failure looks like it is in decoding percent-escapes rather than in which model Z3 returns.

    Repro. boundary.yaml:

    version: 1
    filesystem_policy:
      include_workdir: true
      read_only: [/usr, /lib, /etc]
      read_write: [/tmp]
    landlock:
      compatibility: hard_requirement
    process:
      run_as_user: "1000"
      run_as_group: "1000"
    network_policies:
      archive:
        endpoints:
        - host: gitlab.com
          port: 443
          protocol: rest
          enforcement: enforce
          rules:
            - allow:
                method: GET
                path: /zzz
        binaries:
          - path: /usr/bin/curl

    candidate.yaml: the same file with the rule's path /a%2Fb.

    $ openshell-prover check candidate.yaml --boundary boundary.yaml --output json
    {"result": "inconclusive", "reason_code": "invalid_witness", "reason": "solver returned a model that could not be decoded", ...}
    candidate path result
    /a%2Fb, /a%41b, /a%b inconclusive, invalid_witness
    /a%25b exceeds_boundary, path=/a%25b
    /a_b exceeds_boundary, path=/a_b

    Expected. exceeds_boundary with the counterexample path as written (/a%2Fb), and the same for %41 and a bare %. Related: #4672 (the prover and allow_encoded_slash), where rule paths with %2F are what such policies need.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    state:triage-neededOpened without agent diagnostics and needs triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions