Skip to content

transport: fix zero-state reconnect duplicate delivery; add P formal verification - #405

Open
cbrewster wants to merge 1 commit into
mainfrom
cbrewster/push-muuqnpsxpoom
Open

transport: fix zero-state reconnect duplicate delivery; add P formal verification#405
cbrewster wants to merge 1 commit into
mainfrom
cbrewster/push-muuqnpsxpoom

Conversation

@cbrewster

Copy link
Copy Markdown
Member

Why: a client that sent messages but never received an ack is indistinguishable
from a new session at handshake time, so reconnecting to a server that lost the
session (restart or grace expiry) replayed the send buffer and re-executed
handlers while the original calls hung forever.

What changed:

  • Handshake requests now carry expectedSessionState.isReconnect (optional,
    wire-compatible; sessions track hadConnection across transitions), and the
    server rejects marked reconnects to unknown sessions with
    SESSION_STATE_MISMATCH, yielding the documented hard-reconnect semantics
    (UNEXPECTED_DISCONNECT, fresh session, no replay). Deterministic regression
    test in tests/zerostate.test.ts; PROTOCOL.md documents the field.
  • Adds verification/ (P language, Nix-packaged toolchain): a model-checked
    transport/session model with fault injection (which found this bug and
    reproduces d7c0ec9 as a must-fail regression), an inductive UCLID5 proof of
    the seq/ack sliding window, and PObserve runtime conformance checking of the
    hegel property-suite traces against the model. npm run model:check /
    model:observe; not wired into CI.

Staging test: run two server instances behind a flaky link (or kill/restart one
mid-call with heartbeats disabled); a client with an in-flight rpc against the
restarted server must get UNEXPECTED_DISCONNECT and the handler must not run
twice for the same request. Locally: npx vitest run __tests__/zerostate.test.ts,
nix develop .#verification --command verification/p/check.sh && verification/p/observe.sh.

@cbrewster

Copy link
Copy Markdown
Member Author

@cbrewster
cbrewster requested a review from a team as a code owner August 27, 2026 22:58
@cbrewster
cbrewster requested review from daweifeng-replit and removed request for a team August 27, 2026 22:58

@lhchavez lhchavez left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need to read it in depth, but it looks fantastic

@@ -0,0 +1,156 @@
package river.pobserve;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@replit/zerg rewrite in Kotlin, make no mistakes

@cbrewster
cbrewster force-pushed the cbrewster/push-muuqnpsxpoom branch from 8c704b4 to ef485ee Compare August 28, 2026 00:18
…verification

Why: a client that sent messages but never received an ack is indistinguishable
from a new session at handshake time, so reconnecting to a server that lost the
session (restart or grace expiry) replayed the send buffer and re-executed
handlers while the original calls hung forever.

What changed:
- Handshake requests now carry `expectedSessionState.isReconnect` (optional,
  wire-compatible; sessions track `hadConnection` across transitions), and the
  server rejects marked reconnects to unknown sessions with
  SESSION_STATE_MISMATCH, yielding the documented hard-reconnect semantics
  (UNEXPECTED_DISCONNECT, fresh session, no replay). Deterministic regression
  test in __tests__/zerostate.test.ts; PROTOCOL.md documents the field.
- Adds verification/ (P language, Nix-packaged toolchain): a model-checked
  transport/session model with fault injection (which found this bug and
  reproduces d7c0ec9 as a must-fail regression), an inductive UCLID5 proof of
  the seq/ack sliding window, and PObserve runtime conformance checking of the
  hegel property-suite traces against the model. `npm run model:check` /
  `model:observe`; not wired into CI.

Staging test: run two server instances behind a flaky link (or kill/restart one
mid-call with heartbeats disabled); a client with an in-flight rpc against the
restarted server must get UNEXPECTED_DISCONNECT and the handler must not run
twice for the same request. Locally: `npx vitest run __tests__/zerostate.test.ts`,
`nix develop .#verification --command verification/p/check.sh && verification/p/observe.sh`.
@cbrewster
cbrewster force-pushed the cbrewster/push-muuqnpsxpoom branch from ef485ee to 1e77f1e Compare August 28, 2026 15:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants