fix: async engine answers callers after their macrostep and hands the loop over on cancellation

[fgmacedo]

Why

Two defects in the async engine's processing_loop, both present on v3.2.1 and on main after #688. #689: an exception raised in the macrostep after the lock holder's own event succeeded was lost, and await send() returned the result. #690: cancelling the coroutine running the loop left every queued send() pending until some later send() ran the loop again.

Fixes #689
Fixes #690

What changes

• Async engine with catch_errors_as_events=False: a caller's future is answered when the macrostep of its event ends. An exception from the internal events or eventless transitions that follow the event reaches that caller, as the sync engine raises it from send(). This also applies to a queued caller: in the fix: answer callers queued behind a macrostep that raises #688 test test_failure_after_a_queued_event_..., that caller used to get "crossed" and now gets the exception.
• When no coroutine awaits the failing event (sent from a callback, its caller cancelled while awaiting it, or the lazy __initial__ activation), the coroutine running the loop raises the exception. It used to be dropped silently. Events still queued are dropped and their callers rejected, as in fix: answer callers queued behind a macrostep that raises #688. StateChart is unchanged.
• Cancelling (any BaseException) the coroutine running the loop now stops only that coroutine. The internal queue is cleared, the lock is released, and a coroutine awaiting a queued event takes the loop over (woken via _lock_waiters), so each queued caller gets its own result, delayed events included.
• The caller of the event being processed when the cancellation hit gets StateMachineError("The task running the processing loop was cancelled while it processed this event."). I chose an Exception rather than CancelledError because a CancelledError in a task that nobody cancelled is swallowed by asyncio.TaskGroup and is missed by except Exception. The event cannot be re-run, since its callbacks stopped halfway.
• The cancelled coroutine's own future is cancelled. Its event, if still queued, is processed by whoever runs the loop next, as for a waiter that stops awaiting.
• Docs: a new release-3.2.2-async-loop section in docs/releases/3.2.2.md, which includes a doctest of the Async: an exception in the macrostep after the lock holder's own event is silently lost #689 example. One "behavior changes" bullet is added. The fix: answer callers queued behind a macrostep that raises #688 sentence "the coroutine whose event enabled the failing transitions gets its own event's result" is corrected. docs/async.md is updated.

Evidence

The issue examples were run with uv run python -I repro.py, the script built from the snippets in #689 and #690:

== v3.2.1 (worktree of tag v3.2.1)      == main (c3354bb)                 == this branch
689 async: went {b}                     689 async: went {b}               689 async raised: ValueError('kaboom') {b}
689 sync raised: ValueError('kaboom')   689 sync raised: ValueError(...)  689 sync raised: ValueError('kaboom') {b}
690 other raised: TimeoutError()        690 other raised: TimeoutError()  690 other: stepped
690 config: {a} locked: False           (same)                            690 config: {a} locked: False

• On v3.2.1, the queued caller from Async: cancelling the task that holds the processing lock leaves queued send() callers pending #690 is answered only by a later send(): pending: True steps: 0, then later: 2 other: 1.
• New tests in tests/test_async_futures.py, all with pytest.mark.timeout(5). TestMacrostepErrorReachesTheCallerOfItsEvent runs on both engines via sm_runner. TestCancelledLockHolder covers queued callers, a delayed event, the interrupted caller, a cancelled waiter, a cancelled activate_initial_state(), and that the machine stays usable afterwards. With statemachine/ reverted to main, 17 of them fail, including the 13 async Async: an exception in the macrostep after the lock holder's own event is silently lost #689 cases and all 4 takeover cases, plus the release-notes doctest. With the fix, they all pass.
• timeout 120 uv run pytest -n 4 --cov=statemachine --cov-branch: 2777 passed, 25 skipped, 44 xfailed, TOTAL 100%. ruff check, ruff format --check, mypy --namespace-packages --explicit-package-bases statemachine/ tests/ and pyright statemachine/ are clean. Pre-commit hooks ran on every commit.
• Sync engine: an exception after the caller's own event already raises from send() there, and both engines are covered by test_caller_gets_the_exception[sync-*].
• Two adversarial review rounds, each by a fresh agent:
  • Round 1: the runner raised while its own future had been rejected, so asyncio logged "Future exception was never retrieved". Fixed, and covered by test_caller_running_the_loop_retrieves_its_rejected_future. CancelledError for the interrupted caller was swallowed by TaskGroup, so it became StateMachineError. Docs and test nits were also fixed.
  • Round 2: no code findings. A 400-seed fuzz run (concurrent senders, delays, failures, random holder and waiter cancellation) showed no hangs, no lock left held and no unretrieved futures. Three doc overclaims were narrowed.

Not verified

• Python 3.10 and 3.11 were not run locally (3.12 only). The StateMachineError path does not depend on CancelledError message propagation, but CI is the first run on those versions.
• A waiter now needs one more event-loop tick to observe its answer, because it waits on asyncio.wait for its future or the lock release. 2000 concurrent sends were measured about 5-10% slower in the review run (0.155 s vs 0.145 s), with no benchmark in CI. tests/test_cancel_event.py::test_a_caller_awaiting_a_cancelled_event_gets_none counted ticks and now waits for the task.
• Pre-existing and out of scope, confirmed identical on main:
  • The sync engine does not clear the queue when transition selection of a queued external event raises (a validator, or TransitionNotAllowed), and it keeps internal events after a BaseException.
  • A StateChart cancelled inside an on callback can be left with an empty configuration, because the microstep restores the configuration only on Exception.
  • A caller cancelled before its send() coroutine starts leaves a future nobody awaits.
• Driving one async-engine machine from two event loops in different threads is not covered; it was already unsafe for future resolution.

[sonarqubecloud]

Quality Gate passed

Issues
10 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (c3354bb) to head (5859fa3).

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #691   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           53        53           
  Lines         5859      5897   +38     
  Branches       936       940    +4     
=========================================
+ Hits          5859      5897   +38     

Flag	Coverage Δ
unittests	100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.