WSL2 (kernel 6.6.87, Docker Engine): sandbox create fails with "seccomp notification probe: notification launcher disappeared" #3842

Activity

1. github-actions

   added

   state:triage-neededOpened without agent diagnostics and needs triageOpened without agent diagnostics and needs triage

   on Sep 29, 2026
2. PaulTiffany commented on Sep 29, 2026

   

   PaulTiffany

   on Sep 29, 2026

   More actions

   Independent reproduction on WSL2, via Docker Desktop rather than Docker CE-in-WSL

   We hit this on a different Docker configuration from the original report, which may help narrow the boundary.

   Environment

   • Windows 11, WSL2 (Ubuntu 24.04), kernel 6.6.87.2-microsoft-standard-WSL2
   • Docker Desktop 29.8.0 (the original report used Docker Engine inside WSL)
   • OpenShell v0.1.2, installed from the published .deb
   • docker info security options: name=seccomp,profile=builtin, name=cgroupns

   Failure
   ControlSupervisorStartFailed: Docker sandbox exited before supervisor became ready;
   sandbox log tail: Error: × seccomp notification probe
   ╰─▶ notification launcher disappeared

   Reduced reproduction. We first saw this with a policy and a provider attached, but it reproduces with neither:
   openshell sandbox create --name sb-probe --from --detach -- sleep 120
   No --policy, no --provider, a plain local image. Same error. So it is not policy-, provider- or image-specific.

   Otherwise healthy. openshell doctor check reports all checks passed. openshell status shows the gateway Connected and Authenticated over mTLS, and stays healthy across the failure. openshell sandbox delete cleans up correctly; no container or forward is left behind.

   Not the <5.19 kernel gate. 6.6.87 is well past the WAIT_KILLABLE_RECV fallback added in #3420, so this looks like a separate path.

   Since the original report used Docker CE inside WSL and we used Docker Desktop, the common factor is the WSL2 kernel rather than a particular Docker installation. Happy to run diagnostics if that would help.
3. fede-kamel commented on Sep 29, 2026

   

   fede-kamel

   on Sep 29, 2026

   Contributor

   More actions

   One more data point that may help narrow this down: on a newer WSL kernel, the seccomp notification probe passes with the same OpenShell release and the same Docker flavor.

   • Windows 11, WSL 2 (Ubuntu, NAT networking), kernel 6.18.x-microsoft-standard-WSL2
   • Docker Desktop, engine 29.4.1
   • OpenShell 0.1.2 (CLI and gateway from the release assets), Docker driver

   Here the sandbox container starts and the supervisor runs normally. Sandboxes reach Ready, and policy enforcement, credential placeholders, and the advisor all work end to end. The only thing I needed was the Docker Desktop grpc_endpoint workaround from #3880, which is a separate, later failure.

   So on WSL, both reports here are on 6.6.87 and 6.18 is fine. That points at the WSL 6.6 kernel rather than Docker CE vs. Docker Desktop. Updating WSL (wsl --update) to a 6.18 kernel may be a workaround for the original reporters.
4. russellb commented on Oct 1, 2026

   

   russellb

   on Oct 1, 2026

   Contributor

   More actions

   I checked whether this is a regression of the <5.19 WAIT_KILLABLE_RECV fallback from #3417 / #3420. It is not — but the error message is actively hiding the real cause, which is why the two look identical.

   The #3420 fallback is intact in 0.1.2

   Diffing the full fix surface from 293fab7 through v0.1.2 and through main:

   • install_listener still retries with a plain NEW_LISTENER on EINVAL — byte-identical to what was merged (the function moved to openshell-isolation-interface in the isolation-interface refactor, but was not otherwise touched).
   • The launch invariant is still relaxed to cancellation || task_memory_writes_disabled, with its regression test still present.
   • The legacy_read_only / task_memory_writes_disabled evidence plumbing is unchanged.

   The only production changes to that file since the fix are strictly relaxing or unrelated: #3574 removed the probe_task_memory_copy() self-memory probe, and the workload listener's notified-syscall set gained open/openat/openat2 — which is installed after qualification and so cannot produce this error.

   Image lag is also ruled out: the supervisor image tag is pinned to the crate version, so a 0.1.2 CLI pulls supervisor:0.1.2, built from the tag that contains the fix.

   Why it looks like the same bug

   notification launcher disappeared discards the real errno. In probe_scalar_round_trip:

   let launcher = thread::spawn(move || -> io::Result<libc::c_long> {
       let listener = install_listener(&[libc::SYS_getppid])?;   // real error dies here
       ...
   });
   let (listener, wait_killable) = receiver
       .recv()
       .map_err(|_| io::Error::other("notification launcher disappeared"))?;

   Any failure inside install_listener drops the sender, recv() fails, and the ? detaches the thread without join() — so the launcher's io::Error is never recovered. At least three distinct failures collapse into this one string:

   • verify_notification_sizes — kernel/userspace seccomp_notif struct size mismatch
   • set_no_new_privileges — prctl(PR_SET_NO_NEW_PRIVS) rejected
   • seccomp(SET_MODE_FILTER, NEW_LISTENER) with any non-EINVAL errno (EPERM, EACCES, ENOSYS, ...)

   probe_addfd_send and probe_connected_sendto_fast_path have the same pattern. This predates #3420 — the message dates to the original RFC 0012 implementation in #2942, which is also why #3417 needed reporter-side investigation to pin down.

   So the 6.6.87 WSL2 failure is one of the above, and we currently cannot tell which from the log. Given that 0.1.2 works on WSL 6.18 with the same Docker flavor, a kernel-ABI or kernel-config difference in the WSL 6.6 series is the leading hypothesis, but that needs the errno to confirm.

   Next step

   I'm opening a PR that joins the launcher thread on the recv() failure path and propagates its real error, for all three probes. That turns this into the actual errno and will immediately separate a WSL kernel-ABI rejection from a Docker seccomp-profile EPERM from a genuine flag EINVAL.

   @PaulTiffany — once that lands, rerunning openshell sandbox create on the 6.6.87 kernel should report the underlying error directly. If you want to confirm sooner, running this inside the sandbox image on that host would narrow it the same way:

   python3 -c "import ctypes,os;l=ctypes.CDLL(None);print('nnp',l.prctl(38,1,0,0,0),ctypes.get_errno())"
5. vehystrix commented on Oct 2, 2026

   

   vehystrix

   on Oct 2, 2026

   Last edited by vehystrix

   More actions

   Reproduced on Windows 11, WSL 3.0.1.0, kernel 6.18, docker-ce 29.8.1, Ubuntu 26.04.1 LTS, mirror networking. Same error on all 3 v0.1.x versions.

   And now the crazy part: My Windows 10, WSL 3.0.1.0, kernel 6.18, docker-ce 29.8.1, Ubuntu 24.04.5 LTS, NAT networking, works fine on all 3 v0.1.x versions...

   I'm afraid to update my Ubuntu version now...

   I also ran the python command above on the Win11 machine with the error inside the sandbox image when run directly with docker run, and go the output nnp 0 0
6. vishwasaidev-dev commented on Oct 4, 2026

   

   vishwasaidev-dev

   on Oct 4, 2026

   Author

   More actions

   Ran russellb's diagnostic, then extended it to probe the actual seccomp(SET_MODE_FILTER, NEW_LISTENER, ...) call directly (not just the prctl(NO_NEW_PRIVS) precondition), since vehystrix's nnp 0 0 result on their failing machine suggested that alone wasn't the blocker. Got the identical nnp 0 0 here too.

   Environment: Windows 11 Pro, WSL2 kernel 6.6.87.2-microsoft-standard-WSL2, Ubuntu 24.04.4 LTS, Docker CE 29.8.1 installed directly in WSL (not Docker Desktop), OpenShell 0.1.2 — same machine as the original report.

   Script: sets PR_SET_NO_NEW_PRIVS, builds a minimal valid single-instruction BPF filter (BPF_RET_K / SECCOMP_RET_ALLOW), then calls syscall(SYS_seccomp=317, SECCOMP_SET_MODE_FILTER=1, SECCOMP_FILTER_FLAG_NEW_LISTENER=0x8, &prog) directly via ctypes — the exact call install_listener makes.

   Result, identical on the bare WSL host and inside a fresh nvcr.io/nvidia/base/ubuntu:24.04 container (the same image our original report's failed sandbox used):

   nnp: 0 0
   seccomp_new_listener_fd: -1 errno: 16 Device or resource busy
   

   EBUSY on SECCOMP_SET_MODE_FILTER with NEW_LISTENER is a specific, documented kernel condition: the calling task already has a notification listener attached via an earlier filter in its chain, and the kernel refuses to attach a second one.

   So I checked whether a filter already exists before anything of ours runs:

   $ grep -i seccomp /proc/1/status                                # PID 1 / systemd, bare WSL host
   Seccomp: 2
   Seccomp_filters: 1
   
   $ bash -c 'grep -i seccomp /proc/self/status'                    # fresh shell, bare host
   Seccomp: 2
   Seccomp_filters: 1
   
   $ docker run --rm nvcr.io/nvidia/base/ubuntu:24.04 grep -i seccomp /proc/self/status   # fresh container
   Seccomp: 2
   Seccomp_filters: 2
   

   On this WSL2 environment, a seccomp filter (mode 2 = FILTER) is already attached at PID 1 before systemd starts anything else, and it's inherited by every descendant process — one more gets stacked by Docker's own default profile inside a container. Seccomp filters are inherited across fork/exec and can only be added to, never removed. If that pre-existing PID-1 filter carries a NEW_LISTENER-attached fd from somewhere in the WSL2 boot/interop chain (plausible — Microsoft's WSL2 interop layer is known to lean on seccomp/ptrace-style syscall interception for things like Windows-binary interop), it would explain exactly this EBUSY: OpenShell's supervisor would be the second caller trying to attach a listener to a task that already has one from earlier in the process tree, not a 5.19-style EINVAL or a plain permissions rejection.

   I can't see inside WSL2's closed-source init/interop layer to confirm what installed the first filter, but this gives a precise, reproducible errno plus supporting evidence instead of the swallowed generic message. Happy to run anything else that'd help narrow it further, or to test once the error-propagation PR lands.
7. vehystrix commented on Oct 5, 2026

   

   vehystrix

   on Oct 5, 2026

   More actions

   I did some more digging and it all traces back to mirrored networking mode:
   microsoft/WSL#9548

   My win10 machine uses the default NAT networking mode because mirrored is only supported on Win11
   I just disabled mirrored networking mode on my Win11 machine, and everything just works!

   Just to be sure, I installed openshell v0.0.116 again. With network mode mirrored (and the seccomp_filters set to 1 on the host) I can create a new sandbox without issue. So this feels like a regression from v0.0.116 to v0.1.0
8. vishwasaidev-dev commented on Oct 9, 2026

   

   vishwasaidev-dev

   on Oct 9, 2026

   Author

   More actions

   Confirming vehystrix's root-cause finding independently on the original reporting machine (same environment as the top of this issue: Windows 11 Pro, WSL2, Ubuntu 24.04.4, Docker CE 29.8.1, OpenShell 0.1.2).

   Switched .wslconfig from networkingMode=mirrored to networkingMode=nat, full wsl --shutdown + restart to apply:

   # before (mirrored):
   $ grep -i seccomp /proc/1/status
   Seccomp_filters: 1
   
   # after (nat):
   $ grep -i seccomp /proc/1/status
   Seccomp_filters: 0
   

   With the PID-1 filter gone, openshell sandbox create works cleanly for the first time on this machine. Ran the full lifecycle to be sure it's not just the trivial case:

   • sandbox create --no-keep -- echo ... → succeeds, prints output, cleans up
   • sandbox create --detach (persistent) → reaches Ready
   • sandbox exec -n <name> -- sh -c 'echo first-task > /tmp/state.txt' then a second separate exec reading it back → state persists across calls, as expected for the reuse-not-recreate pattern
   • sandbox stop → sandbox start → state file still there → preserved across stop/start as documented
   • sandbox delete → clean teardown

   So on this machine at least, this is a clean, complete fix — not a partial workaround. Matches vehystrix's finding exactly: mirrored networking's own seccomp-notify filter on PID 1 (installed to intercept bind() for inbound connectivity) is what's colliding with OpenShell's supervisor trying to install its own, via the single-listener-per-task kernel rule. Trading away mirrored mode's inbound-connectivity benefit is a real cost for anyone relying on it, but at least it's now a known, reliable lever rather than a mystery.

   Given this + the earlier EBUSY/Seccomp_filters data point (comment above), the explanation in microsoft/WSL#9548 (filter installed on PID 1 specifically for mirrored mode's bind() interception, one listener per task) accounts for everything observed in this thread. The open question left is vehystrix's separate note that v0.0.116 apparently worked fine with mirrored mode on — if that holds up, there may also be a secondary regression in how 0.1.x's supervisor requests its listener (e.g. new reliance on NEW_LISTENER where an older version didn't need one), independent of the WSL-level conflict. Not something I can test myself since I don't have an old 0.0.116 install handy, but flagging it in case that's a separate, fixable half of this.
9. vehystrix commented on Oct 9, 2026

   

   vehystrix

   on Oct 9, 2026

   More actions

   To check with 0.0.116 I just did:

   sudo apt uninstall openshell
   curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | OPENSHELL_VERSION=0.0.116 sh
   

   I did have to temporarily rename my gateway.toml file (0.0.116 doesn't support v2 of the file), but what's in the file doesn't matter to reproduce this issue.