Skip to content

feat(runtime): sendable actor heaps and the explicit execution context - #3

Merged
niklas-heer merged 12 commits into
mainfrom
feat/sendable-actor-heaps
Sep 16, 2026
Merged

niklas-heer merged 12 commits into
mainfrom
feat/sendable-actor-heaps

Conversation

@niklas-heer

Copy link
Copy Markdown
Collaborator

Step 1 of five toward parallel actor execution, specified in
docs/superpowers/specs/2026-09-15-sendable-actor-heaps-design.md and planned in
docs/superpowers/plans/2026-09-15-sendable-actor-heaps.md.

Why

The scheduler executes FIFO continuation callbacks on one thread, and the BEAM
comparison records actor throughput, scheduler fairness and fault recovery as
unmeasured. Reaching parallelism means changing heap ownership, which is the
hardest thing in the runtime to change once more code depends on it.

This step produces no parallelism. It should be judged on whether it makes step 2
possible without a second ownership rewrite.

What changed

Heap ownership moves out of thread_local! { static STORE } into an explicit
Domain. The thread-local keeps only a cursor to the domain executing on this
thread, installed by an Activation guard that restores the previous value on
drop. Domain is Send; Root and Scope stay thread-bound.

Foreign entry point signatures and the compiler's symbol contract are unchanged.
The 257 no_mangle entry points reach the store through roughly twelve functions
in memory.rs, so nothing outside memory/heaps.rs changed shape and no codegen
work was required.

Commits 1-5 are a mechanical extraction that changes no ownership. Commit 6 is the
only ownership change. Reviewing them in order is much easier than reviewing the
diff as a whole.

Evidence

  • The seeded actor scenario reproduces trace hash 01a55a0046de5614 before and
    after, with identical 48,334 callbacks, 1,226 delivered, 3,774 timeouts, 2,501
    restarts and 9,977 churn actors, and zero cleanup residue. Record and replay
    round-trips.
  • Full macOS ARM64 gate: 316 native-output fixtures, 20 examples, 63 dynamic
    compatibility programs, 295 atomic rejections, 64+192+231 fuzz cases.
  • Runtime suite grows 99 to 107 tests, including a cross-heap edge oracle whose
    negative case fabricates an edge the collector would never build, so the oracle
    is demonstrably able to fail.
  • ThreadSanitizer runs the instrumented suite clean: 107 tests, zero race reports,
    2,389 s on one Apple M4.

Measured cost

The allocation-heavy immutable model (workloads.mr model 100000) regresses from
a 59.03 ms to a 59.96 ms median, +1.58%, over 25 interleaved samples per build.
The allocation-light scalar control moves +0.09%, which locates the cost in the
allocation path: the un-activated default path now takes two thread-local accesses
where it took one. libmorrow_runtime.a grows 13,494,480 to 13,506,976 bytes.

The regression is recorded rather than rounded off. Step 2 runs every actor under
an activated domain, making the cursor path hot instead of the default path;
whether that recovers the difference is a measurement for that step.

Worth reviewing closely

  1. The Busy aliasing guard. Replacing RefCell with a raw pointer would
    silently drop the aliasing check it enforced. Root::drop can run from a
    finalizer during collection, re-entering the cursor while collect_active
    holds the domain. That was a panic; without the guard it becomes undefined
    behaviour.
  2. collect_active scans every actor heap in the domain when the invocation
    heap is active. The design document missed this. It is a collection-time
    coupling, and it means a scheduler's domain must own its actor heaps rather
    than lend them out, which constrains step 2.
  3. Domain: Send is asserted at compile time; Root/Scope staying !Send is
    not.
    Rust cannot express a negative auto-trait bound without fragile tricks,
    so that half rests on the PhantomData<Rc<()>> markers.

Note on the new CI job

The ThreadSanitizer job takes about 40 minutes and finds nothing while execution
is single threaded. It is added now so the harness and any suppressions exist
before parallel schedulers make it load bearing. Restricting it to pushes on
main rather than every pull request is a reasonable call.

🤖 Generated with Claude Code

niklas-heer and others added 12 commits September 15, 2026 03:31
The scheduler runs on one thread and the BEAM comparison records actor
throughput and scheduler fairness as unmeasured. Reaching parallelism means
changing heap ownership, which is the hardest thing in the runtime to change
once more code depends on it.

The spec sequences five required steps and designs the first: heap ownership
moves out of thread-local storage into actors and domains, leaving the
thread-local as a scheduling cursor. Foreign entry point signatures stay
unchanged, which keeps this step independent of the codegen work preemption
will need.

It records two findings from reading the tree. Per-actor heap isolation already
exists, since Store holds a map of slots each owning a Heap behind an active
cursor. The obstacle to migration is therefore not the !Send marker but the
shared invocation heap that actor payload heaps hold control edges into.

Determinism becomes a property of the simulation driver rather than of the
runtime, because a BEAM-shaped runtime cannot offer bit-exact replay of a real
multi-threaded run. ThreadSanitizer enters the gate here, before step 2 makes
it load-bearing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Ten TDD tasks implementing step 1 of the parallel actor work. Tasks 1 to 5
extract the thread-local store into an explicit Domain without changing
ownership, so the mechanical work is reviewable on its own. Task 6 reduces the
thread-local to a cursor, which is the only ownership change. Tasks 7 to 10
add the type-level guard, the cross-heap edge oracle, ThreadSanitizer and the
measurements the decision record needs.

Reading the tree while planning corrected the spec twice. The 257 runtime entry
points reach the store through roughly twelve functions in memory.rs, so no
entry point signature or call site outside memory/heaps.rs changes. And
invocation-heap collection scans every actor heap in the domain, a sharper
coupling than the control edges the spec identified, which constrains the
scheduler design in step 2.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ask 2 test

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ation path

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Root::drop still resolves through the thread-local store, so the test retires
its token explicitly rather than through Drop. Task 6 makes the cursor
authoritative and closes that gap.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Widen Heap to pub(crate) so Domain's crate-visible accessors can name it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
collect_active keeps the active==0 branch that scans every actor heap in the
domain for control words. That loop is why a domain must own its actor heaps
rather than lend them out, which constrains the scheduler design in step 2.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The thread-local no longer owns heaps. It holds a pointer to the domain
executing on this thread, installed by an Activation guard that restores the
previous value on drop, so the invariant survives unwinding and fault paths.
A domain built on one thread can now be moved to another and serve the
ordinary allocation path there.

Replacing RefCell with a raw pointer would have silently dropped the aliasing
check RefCell enforced, turning a would-be panic into undefined behaviour, so
the cursor path keeps an explicit re-entrancy guard.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…enarios

verify_edges reports any managed edge leaving an actor payload heap for
something other than invocation control storage, which is the reference step 4
has to sever. A negative test fabricates an edge the collector would never
build, so the oracle is demonstrably able to fail rather than vacuous.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The shipped standard library is not instrumented, so -Zsanitizer=thread alone
fails the sanitizer ABI check. The command rebuilds std from source and detects
the host triple, which the sanitizer requires to be explicit.

Route remove_root through the same aliasing guard as with_current. A finalizer
running during collection can drop a Root, which re-enters remove_root while
collect_active holds the domain. RefCell made that a panic; the raw cursor would
have made it undefined behaviour.

The instrumented suite takes about 40 minutes and finds nothing while execution
is single threaded. It joins the gate now so the harness exists before parallel
schedulers make it load bearing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Decision156 records the ownership change, the two findings the work produced and
the measured cost. The allocation-heavy immutable model regresses 59.03 to
59.96 ms, +1.58% over 25 interleaved samples per build, while the scalar control
moves +0.09%. That locates the cost in the allocation path, where the
un-activated default path now takes two thread-local accesses instead of one.
The regression is recorded rather than rounded off; whether step 2 recovers it
by making the cursor path hot is a measurement for that step.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@niklas-heer
niklas-heer merged commit 4daffcf into main Sep 16, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant