all: Add support for OpenBSD - #535
Open
tannevaled wants to merge 2 commits into
Open
tannevaled wants to merge 2 commits into
tannevaled wants to merge 2 commits into
Conversation
Closes ebitengine#454, which has been open since May with the error every consumer hits: purego does not merely lack OpenBSD, it FAILS TO COMPILE there, so any program importing it at all is blocked. Navidrome hit it through gen2brain/webp; go-doom hits it through Ebitengine's GL loader. Everything below was read from a real OpenBSD 7.9/arm64 machine rather than adapted from NetBSD, because "the BSDs agree" is a guess: - RTLD_* from /usr/include/dlfcn.h. They DO agree with NetBSD's; checking was cheaper than finding out they had stopped. - stack_t and SS_DISABLE (0x004) from /usr/include/sys/signal.h. - PTHREAD_MUTEX_INITIALIZER and PTHREAD_COND_INITIALIZER are both NULL (pthread.h:157-158), so uintptr(0) is right. - dlopen/dlsym/dlerror/dlclose are weak symbols in libc; there is no libdl, so `#cgo !netbsd LDFLAGS: -ldl` becomes `!netbsd,!openbsd`. Without that the cgo build fails with "unable to find library -ldl". - pthread is NOT in libc: pthread_create is absent from libc.so.103.0 and lives in libpthread.so.28.1, so the generator names libpthread.so for it. - libc exports environ and __progname, and does NOT export __ps_strings, which NetBSD's equivalent file supplies. The soname deserves its own note, because it is the thing that looks like a blocker and is not. OpenBSD ships no unversioned libc.so FILE -- /usr/lib holds libc.so.103.0 and nothing else. "libc.so" is still what belongs in cgo_import_dynamic: the Go runtime itself writes exactly that in runtime/sys_openbsd.go. Verified at runtime too, which is the part that could have gone either way: Dlopen("libc.so") and Dlopen("libc.so.103.0") return the SAME handle, because ld.so resolves the soname. What passes, natively, CGO_ENABLED=0, on openbsd/arm64: Dlopen, Dlsym, nested Dlopen, SyscallN, SyscallN's errno handling, RegisterFunc including the concurrent pointer-return case, floats, and qsort -- which matters because qsort takes a CALLBACK, so NewCallback works. RegisterLibFunc binds and calls strlen and puts for real. ⚠ ONE TEST STILL FAILS and this is not ready to merge because of it. TestRegisterLibFunc_Bool dies with SIGILL. It is the one case that calls a callback DIRECTLY from Go rather than letting C call it. This CPU reports BT (Branch Target Identification) and PAC in its feature line, and SIGILL is what BTI raises on an indirect branch to an instruction that is not a landing pad -- but that is a hypothesis with a plausible mechanism, not a diagnosis, and it is recorded as such. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Closes ebitengine#454. purego does not merely lack OpenBSD: it FAILS TO COMPILE there, so any program that imports it at all is blocked even on a path it never takes. struct_<arch>.go and cdecl.go carry no OS constraint -- only the filename's GOARCH suffix -- while the symbols they use live in syscall.go behind (darwin||freebsd||linux||netbsd||windows). That is the error in ebitengine#454, and the one Navidrome hit through gen2brain/webp. Everything here was read from a real OpenBSD 7.9/arm64 machine rather than adapted from NetBSD, because "the BSDs agree" is a guess: - RTLD_* from /usr/include/dlfcn.h. They DO agree with NetBSD's. - stack_t and SS_DISABLE (0x004) from sys/signal.h. - PTHREAD_MUTEX_INITIALIZER and PTHREAD_COND_INITIALIZER are NULL. - dlopen and friends are weak symbols in libc and there is no libdl, so `#cgo !netbsd LDFLAGS: -ldl` becomes `!netbsd,!openbsd`. Without it cgo fails with "unable to find library -ldl". - pthread_create is NOT in libc.so.103.0; it is in libpthread.so.28.1. - libc exports environ and __progname, and no __ps_strings, which the NetBSD file supplies. The soname looks like a blocker and is not. OpenBSD ships no unversioned libc.so FILE. "libc.so" is still right: the Go runtime writes exactly that in runtime/sys_openbsd.go, and at RUNTIME Dlopen("libc.so") and Dlopen("libc.so.103.0") return the same handle, because ld.so resolves the soname. BTI is the part that needed its own file. OpenBSD/arm64 enforces Branch Target Identification, and the shared zcallback_arm64.s cannot work under it. callbackasmAddr computes entry i as base+i*8; the assembler emits a `BTI c` at the TEXT symbol, so base IS a landing pad and entry 0 works by accident, while entry 1 lands on entry 0's B -- not a landing pad -- and raises SIGILL. Measured before fixing: callback 0 at 0xc3640 returned, callback 1 at 0xc3648 died, and objdump showed a JMP at that address. The same program on darwin/arm64 returns all four callbacks correctly, which is what says this is OpenBSD's enforcement rather than a general arm64 bug. So zcallback_openbsd_arm64.s gives every entry its own landing pad -- three instructions, entrySize 12 for openbsd/arm64 only. The first entry deliberately has none written for it: the assembler's own pad at the symbol serves as it. Writing a second made entry 0 sixteen bytes while the rest were twelve, which moved the fault rather than removing it; that intermediate state was measured too. Verified natively on openbsd/arm64, CGO_ENABLED=0: the whole test suite passes. darwin/arm64 still passes. linux 386/arm/amd64/arm64, netbsd, windows, darwin and openbsd/amd64 all still cross-compile. Two honest limits. openbsd/amd64 cross-compiles and is NOT tested -- it needs no BTI handling and takes the shared amd64 path, but nobody has run it. And freebsd/amd64 and freebsd/arm64 fail to build on this Go with "//go:cgo_export_dynamic environ only allowed in cgo-generated code"; that reproduces on main without this branch, so it is not from here, but it is worth someone's attention. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Member
|
Add tests for GitHub Actions, thanks |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #454.
purego does not merely lack OpenBSD — it fails to compile there, so any program that imports it at all is blocked, even on a path it never takes.
struct_<arch>.goandcdecl.gocarry no OS constraint (only the filename's GOARCH suffix), while the symbols they use live insyscall.gobehind(darwin||freebsd||linux||netbsd||windows). That is exactly the error in #454.Developed and tested on a real OpenBSD 7.9/arm64 machine, not cross-compiled.
Read from the machine, not adapted from NetBSD
"The BSDs agree" is a guess, so each value was checked:
RTLD_*/usr/include/dlfcn.h— they do agree with NetBSD'sstack_t,SS_DISABLE(0x004)sys/signal.hPTHREAD_{MUTEX,COND}_INITIALIZERNULL, souintptr(0)-ldldl*are weak symbols in libc.#cgo !netbsd LDFLAGS: -ldl→!netbsd,!openbsd, else cgo fails withunable to find library -ldlpthread_createis not inlibc.so.103.0; it is inlibpthread.so.28.1__ps_stringsenvironand__prognamebut no__ps_strings, which the NetBSD file suppliesThe soname looks like a blocker and is not. OpenBSD ships no unversioned
libc.sofile."libc.so"is still correct: the Go runtime writes exactly that inruntime/sys_openbsd.go, and at runtimeDlopen("libc.so")andDlopen("libc.so.103.0")return the same handle, becauseld.soresolves the soname.BTI needed its own callback table
OpenBSD/arm64 enforces Branch Target Identification, and the shared
zcallback_arm64.scannot work under it.callbackasmAddrcomputes entry i asbase + i*8. The assembler emits aBTI cat the TEXT symbol, sobaseis a landing pad — entry 0 works by accident, while entry 1 lands on entry 0'sB, which is not a landing pad, and raises SIGILL.Measured before fixing: callback 0 at
0xc3640returned; callback 1 at0xc3648died;objdumpshowed aJMPat that address. The same program on darwin/arm64 returns all four callbacks correctly, which is what says this is OpenBSD's enforcement rather than a general arm64 bug.So
zcallback_openbsd_arm64.sgives every entry its own landing pad — three instructions,entrySize = 12foropenbsd/arm64only. The first entry deliberately has none written for it: the assembler's own pad at the symbol serves as it. Writing a second made entry 0 sixteen bytes while the rest were twelve, which moved the fault rather than removing it — that intermediate state was measured too.Verification
CGO_ENABLED=0: the whole test suite passes.Two honest limits
//go:cgo_export_dynamic environ only allowed in cgo-generated code. That reproduces onmainwithout this branch, so it is not from here — but it looks worth someone's attention.🤖 Generated with Claude Code