Multi-threaded accept: a reuse-port request that names its promise, and the handoff proven - #83
Merged
Merged
Conversation
added 3 commits
September 16, 2026 13:24
SO_REUSEPORT is spelled the same on Linux and the BSDs and does not mean
the same thing. On Linux it permits the duplicate bind and distributes
incoming connections across the listeners holding the address. On macOS
and the other BSDs it permits the duplicate bind and then gives new
connections to whichever socket bound last.
A bool cannot express that, so ListenOpts::reuse_port and UdpOpts::reuse_port
are now a ReusePort enum: No, Share (duplicate binding, delivery unspecified)
and Distribute (the kernel spreads connections across listeners). Distribute
is SO_REUSEPORT on Linux and Android, SO_REUSEPORT_LB on FreeBSD 12.0+, and
Unsupported everywhere else - refused when the listener is created rather
than answered with plain SO_REUSEPORT, which is what produced a listener that
is never given any work. Measured on macOS 15: two loops sharing one port
under SO_REUSEPORT split 32 connections [0, 32].
AF_UNIX listeners now report Unsupported rather than InvalidInput, matching
the rule the rest of the option surface follows.
Contract tests:
- reuse_port_share asserts the duplicate bind works and that every
connection is accepted by someone, and deliberately asserts nothing
about which listener, because Share promises nothing about it.
- reuse_port_distribute asserts there is no third outcome: a platform
either distributes or refuses the listener.
- reuse_port_refused covers Windows and WASI, which have neither.
- handoff_accept_exactly_once and kernel_accept_exactly_once serve 64
connections across 4 loops on 4 threads by each route and assert every
connection was served exactly once, on loops whose handle ceiling is
low enough that a leaked handle exhausts it.
turnloop-bench --accept-scaling drives N loops on N threads serving one port by either route and reports connections per second with the per-loop service counts beside it, so the distribution is visible next to the total rather than inferred from it. It asserts its own subject ran before it prints anything: every loop must have served at least one connection, the per-loop counts must sum to the budget, and --connections must be at least 100 per loop so a loop cannot be given nothing by chance. A scaling number from a run where three of four loops sat idle would read as a flat curve and mean nothing. --route reuse-port is refused up front, with a status and one line naming the alternative, on the platforms that cannot distribute in the kernel.
DESIGN 5a.6 said the kernel balances on Linux/FreeBSD. That is right for Linux and wrong for FreeBSD, where plain SO_REUSEPORT keeps its original BSD meaning and SO_REUSEPORT_LB is the option that distributes. It also said macOS does not balance without saying what it does instead, which is give the whole port to the last binder. docs/multi-threaded-accept.md is the page: the two routes, the per-platform table, why Windows has only one of them, what the contract tests assert, and the runbook for the scaling harness - including the step that has to come first, which is raising --clients until the number stops moving, because the client side is synchronous and a server-scaling curve measured against a saturated client is a picture of the client. It also says what a result would mean in each direction, so the sweep can be read rather than interpreted. No timing numbers are recorded: the harness has only been run for correctness, on loaded machines, where a figure would be worse than none.
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Advanced Run ID: 📒 Files selected for processing (17)
📝 WalkthroughWalkthroughChangesReusePort and multi-threaded accept
Estimated code review effort: 5 (Critical) | ~90 minutes Change: Feature · Severity of issue fixed: Medium Sequence Diagram(s)sequenceDiagram
participant Client
participant Listener
participant AcceptWorker
participant ConnectionWorker
Client->>Listener: connect
Listener->>AcceptWorker: accept
AcceptWorker->>ConnectionWorker: detach and attach connection
ConnectionWorker-->>Client: echo payload and close
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
A request the platform cannot honour now fails before socket() is called rather than after SO_REUSEADDR has been set on a descriptor that is about to be dropped. Nothing reaches the kernel for a listener that is refused.
The figures were cited against macOS 15 and Linux 6.8; they were taken on macOS 26.5 arm64 and Linux 6.17 x86-64. A measurement whose host is wrong in the record cannot be reproduced or contradicted later. The runbook also claimed QueryProcessCycleTime on Windows. turnloop-bench's counter has a Linux arm and a macOS arm and nothing else, so Windows falls back to elapsed nanoseconds - which is the right unit for a scaling sweep in any case, because connections per second is the quantity of interest and an instruction count attributes only the measuring thread's work.
It is the usual objection to multi-threaded accept and it applies to a shape turnloop does not have: a herd comes from several threads waiting on one listening socket. Under Distribute each loop has its own listener and its own accept queue and exactly one loop wakes; under the handoff route exactly one loop is accepting at all. Each route's real cost is named instead.
The client threads shared the connection budget through a countdown: `try_update(|n| n.checked_sub(1))`, which is a compare-exchange loop that retries whenever two client threads reach for the same connection at the same moment. That contention sits on the load-generating side of a harness whose whole job is to not be the bottleneck while the server scales, and it grows with `--clients` — exactly the knob the runbook says to raise first, until the client rate stops moving, before any server curve means anything. A ticket counter claims the same work with one wait-free `fetch_add`: each thread takes the next number and stops once the numbers run past the budget. Exactly `--connections` tickets are below the budget, so exactly that many connections are driven, which `driven == args.connections` still asserts. Verified on macOS 26.5 arm64 at 1, 2 and 4 loops: the budget is spent exactly and the per-loop counts sum to it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements DESIGN §5a.6, multi-threaded accept: one server on more than one core, by the two routes the design already names. Closes #49.
1.
reuse_portsays which behaviour it is asking forSO_REUSEPORTis spelled the same on Linux and on the BSDs and does not mean the same thing, soListenOpts::reuse_portandUdpOpts::reuse_portare no longer abool:Share— permit the duplicate bind, promise nothing about delivery. The traditional BSD use: multicast/broadcast receivers, and zero-downtime restarts where last-binder-wins is the effect you want.Distribute— permit the duplicate bind and spread incoming connections across the listeners.ShareDistributeSO_REUSEPORTSO_REUSEPORTSO_REUSEPORTSO_REUSEPORT_LB(12.0+)SO_REUSEPORTUnsupportedUnsupportedUnsupportedUnsupportedUnsupportedAF_UNIXlistenersUnsupportedUnsupportedThere is deliberately no third outcome: a platform distributes, or the listener is refused when it is created. The failure that prevents is silent and was measured, not assumed — two loops sharing one port under plain
SO_REUSEPORTon macOS 26.5 split 32 connections[0, 32]. The first listener never accepts anything, ever, with no error anywhere. A host that developed the Linux path and shipped it would have a server that looks correct and uses one core.Two corrections fall out of this. DESIGN said the kernel balances "on Linux/FreeBSD"; that is right for Linux and wrong for FreeBSD, where plain
SO_REUSEPORTkeeps its original BSD meaning —SO_REUSEPORT_LBis the one that distributes, and it is whatDistributenow sets there. And the existing contract test asserted distribution on FreeBSD, which would have failed there; FreeBSD is not in CI, so it had never run.ShareandDistributeare the samesetsockopton Linux, which is fine: a variant states a floor, not a ceiling.It distributes by hash, not by load
The kernel picks a listener by hashing the connection's 4-tuple. Nothing asks how busy a loop is, so a loop inside a long turn keeps being handed its share and those connections wait while another loop is idle. Measured, 64 connections over 4 loops on Linux 6.17:
[7, 12, 21, 24]. The docs say this rather than implying even distribution.2. The handoff works today
detach/attachalready worked; this establishes it with a test that accepts on one loop and then drives reads and writes on a sibling loop on another thread, and documents it as the supported multi-core route for macOS, and the only one for Windows, where a socket joins exactly one completion port permanently.accept_handoff_distributiononmainalready covered the basic shape and passes. What it did not cover is exactly-once delivery and handle orphans, which is what the new tests add.3. Correctness tests
multi_threaded_accept_by_handoffandmulti_threaded_accept_by_reuse_port: 64 connections, 4 loops on 4 threads, one port, by each route.0..64with no duplicate and no gap. A connection dropped at handoff never answers and fails the client's read.max_handles: 8while serving 64 connections. The workload fits comfortably one connection at a time, but a handle leaked per connection — at accept,attach,detachorclose— exhausts the ceiling long before the run ends and turns an invisible orphan into aResourceLimitfailure.!alive()at the end.Measured while writing these (Linux 6.17 x86-64 / macOS 26.5 arm64, both pass):
Share, 2 listeners, 32 conns[17, 15][0, 32]Distribute, 2 listeners, 64 conns[30, 34]Unsupported[7, 12, 21, 24][16, 16, 16, 16][16, 16, 16, 16]4. Scaling harness — built, deliberately not measured
turnloop-bench --accept-scaling --route handoff|reuse-port --loops N --connections C --clients K. One JSON line withconnections_per_secondand, beside it,per_loop— the service count for each loop, so the distribution is visible next to the total rather than inferred from it.It asserts its own subject ran before printing anything: the per-loop counts must sum to the budget, every loop must have served at least one connection, and
--connectionsmust be at least 100 per loop so a loop cannot be given nothing by chance. A flat curve produced by three of four loops sitting idle is the failure mode this makes impossible.--route reuse-portexits 2 with one line naming the alternative where the kernel cannot distribute.No timing numbers are reported. The harness has been run only for correctness, on machines at load ~11/10 cores and ~24/64 cores, where a figure would be worse than none. The runbook in
docs/multi-threaded-accept.mdsays what to run, in what order, and what a result would mean in each direction — starting with the step that has to come first: the client side is synchronous, so its rate is bounded byK / RTT, and--clientsmust be raised until the number stops moving before any server-scaling curve means anything.Does the handoff work today? Yes, including on Windows
Independently of this PR:
main's own CI run for164e8e4showshandoff_distribution ... okontest-native (windows-2025, all-features), alongsidepending_operations_refuse_handoffandclosing_and_closed_handles_refuse_handoff. Accepting on one loop and driving I/O on a sibling loop on another thread already works on every native backend, Windows included. The gap was never the mechanism; it was that nothing asserted exactly-once delivery or checked for orphaned handles, and that no document said this is the route macOS and Windows have to take.Verified
cargo test -p turnloop -p turnloop-contract --no-fail-fast -- --test-threads=1green; fmt, clippy, rustdoc clean.filesystem::permission_denied_is_reported, which fails identically on cleanmainon that host because it runs as root and root bypasses the permission check. Both harness routes run there at 1, 2 and 4 loops with exact per-loop accounting.x86_64-pc-windows-msvc,wasm32-wasip2,wasm32-unknown-unknown,x86_64-unknown-freebsd(theSO_REUSEPORT_LBpath).Distribute→SO_REUSEPORT_LBcompiles for the target and is unverified at runtime, like the rest of the FreeBSD lane. It corrects a mapping that was wrong before; it is not a claim that it has been observed working.handoff_accept_exactly_onceandkernel_accept_exactly_onceare registered in the Windows contract list, sotest-native (windows-2025)is the first run of the exactly-once, handle-ceilinged handoff on the platform where the handoff is the only route.Summary by CodeRabbit
New Features
Documentation
Tests