[niteshbalusu11]

Summary

Each IBD write batch ends with flush_class_c_tip, about 8 drive flushes. The write thread used to wait for it. Now the flush runs on a helper thread and the next batch starts right away. The next commit waits for that flush before it touches Class C, so the order on disk does not change.

 write batch N
   archive, checks, Class C commit
-  flush_class_c_tip                 (writer waits)
+  start flush_class_c_tip on rbtc-tip-flush
 write batch N+1
   archive, checks                   (runs during N's flush)
+  wait for N's flush
   Class C commit

• Tip mode still flushes inline.
• Everything that takes tip_commit waits for the flush first: the next commit, disconnect, rewind, Query::flush, flush_for_shutdown, and dropping the Query.
• If the helper thread cannot start, the commit flushes inline.
• If a background flush fails, every later commit fails until restart. A failed fsync may have dropped dirty pages, so it is not retried. IBD stops after its one retry.
• Store fixes, because tables are now written and flushed at the same time:
  • A write that lands in the flush's "mark clean" step is kept (array tables and strong_tx).
  • A write during a file sync keeps the file marked for the next sync. One flush per file at a time.
  • One flush per array table at a time, so an older copy never lands on top of a newer one (the bad-body repair also flushes header_txs).
• With --prune-seqsigwit, pruning now follows the tip that is already on disk, not the one still being flushed.
• New perf field write.tip_flush_wait: how long a commit waited for the previous flush.

Evidence

Signet IBD from one local node, scripts skipped, fresh datadir each run, order base, new, new, base.

	default flags	all indexes
macOS, this branch	419.7 s → 390.6 s (-6.9%)	585.6 s → 556.5 s (-1.5% to -5%)
Linux, prototype of the same change	458.8 s → 446.5 s (-2.7%)	551.1 s → 530.9 s (-3.7%)

• Every run with the change beat every run without it, on both machines. All runs ended on the same tip hash. On Linux each run also restarted and matched getblock and getblockfilter against the reference.
• macOS writer: class_c went from 109 s to 0.4 s. About half came back as tip_flush_wait (50 s), because the macOS flush takes longer than the next batch.
• The Linux numbers are from the prototype (e353c96). Since then: failure handling, the extra wait points, the store fix, and the perf field. None of these are on the hot path, and macOS gave -7% on both versions.
• Tests:
  • failed_ibd_tip_flush_stops_commits_until_restart: a read-only store dir makes the tip seal fail. Later commits fail, still fail once the dir is writable again, and a restart opens on the last durable tip. Fails on master.
  • disconnect_and_drop_wait_for_the_ibd_tip_flush: a test hook holds the flush; disconnect and drop only return after it is released, then reopen on the right tip.
  • ibd_commit_flushes_inline_when_its_thread_cannot_start: a test hook fails the spawn; the commit is still on disk.
  • set_during_flush_clean_is_flushed_next, write_during_flush_stays_pending, concurrent_flushes_keep_the_newer_image: each store race, forced with a hook. Each fails before its fix.
  • ibd_seqsigwit_prune_follows_the_durable_tip: pruned at 112 before the fix, 12 after (300 on disk, keep 288).
• fmt, clippy, deny, ast-grep, and os-smoke pass locally. Workspace tests pass except two Mac-only tests that also fail on master.

Merge Danger

Door: two-way

Blast Radius: IBD durability

• A crash can lose one more write batch than before. Those blocks are downloaded again. The on-disk order is the same, so the tip is never torn.
• After a crash, that batch's spend annotations and filter or tweak entries can be on disk above the tip. Open already replays annotations and trims both indexes to the tip.
• A failed background flush now stops block confirmation until restart, instead of being retried.
• crash-recovery.md and concurrency.md describe the new order.

🤖 Generated with Claude Code

[rearden-grok]

Summary

The IBD overlap preserves the durability order it claims. confirm_blocks_run_with_create_pins still mutates Class C and publishes the tip, then either fsyncs inline (Tip mode) or hands flush_class_c_tip to rbtc-tip-flush (Direct). The next tip_commit taker joins that thread before it touches Class C, so batch N+1 cannot commit a new tip, strong bits, or confirmed[] while N's barrier is in flight. The barrier itself is unchanged: headers, then strong_tx, header_txs, confirmed[] last, then tip_seal. A kill therefore loses at most the in-flight batch (bodies already dequeued, annotations and index puts possibly ahead of the durable tip); open still clamps an unsealed confirmed[] suffix, a spend marker above the revalidated tip, and filter/tweak rows above that tip. Seqsigwit prune uses the tip whose flush has already been joined, not the batch still in flight.

I would request changes for one error-path hole: a failed rbtc-tip-flush spawn leaves the batch committed in RAM with neither a running barrier nor TipFlush::Failed, and a one-block retry then returns success. The happy-path crash window and the sticky fsync failure do what the PR says. Two new tests do not fail if the race they name is removed.

Issue counts by severity

• bugs: 1
• suggestions: 2
• nits: 0

[niteshbalusu11]

@rearden-grok please re-review the latest commit (cdcde2e).

[rearden-grok]

Summary

The overlap keeps the durability order it claims. lock_tip_commit joins rbtc-tip-flush before the next Class C commit, disconnect, rewind, Query::flush, flush_header_archive, flush_for_shutdown, and Drop. A failed spawn flushes inline, and a failed barrier sticks as TipFlush::Failed. The one-block short-circuit cannot skip that, because the check now runs after the join. The store races match the code: the dirty low-water resets before the generation CAS and is restored if the CAS loses, needs_sync clears before the length is read and comes back on error, and one flusher at a time per array table. Seqsigwit prune uses the pre-batch tip while that flush is still running.

The earlier change request is addressed on cdcde2ee and still holds on this head: the spawn-fail path, the flush-lock test without a trailing flush, and the disconnect/drop hold. The new comments that stay are the counterintuitive ones (clean the low-water before the CAS, clear needs_sync before reading the length, do not retry a failed fsync).

Issue counts by severity

• bugs: 0
• suggestions: 0
• nits: 0

Rebase is not required. GitHub reports this head mergeable. The four master commits since the base record the BIP34 ancestor in RAM after class_c_commit returns, and drop it on disconnect. That does not change Class C flush order.

[rearden-grok]

Summary

The earlier approval on d8fa2f2 still holds. 878ac55 is that series rebased onto 0a1d040, not a new patch. git range-diff of c70b341..d8fa2f2 against 0a1d040..878ac55 is nine commits. Seven are identical. The two range-diff marks are context only: master now calls remember_connected_bip34 immediately after class_c_commit, and the confirm-write row now says a crash inside the era can leave Class A above the tip. GitHub reports this head mergeable.

That rebase does not add a second Class C flush or drop a join. confirm_blocks_run_with_create_pins still takes lock_tip_commit (which joins rbtc-tip-flush) before check_run_extends_tip, the strong bits, and confirmed[]. flush_tip is still the only barrier: inline in Tip mode, spawn_tip_flush in IBD. Disconnect, rewind (rewind_to_height_inner / try_rewind_to_height), Query::flush, flush_header_archive, flush_for_shutdown, and Drop still join. forget_bip34_ancestor_if_reorged runs inside disconnect_tip_body after that join, and the ancestor note is RAM only. The commits that landed under the rebase (BIP34 ancestor, archived same-batch parent, submitblock witness reserved value, Class A above the tip) do not open another flush_class_c_tip path. The tip-hole cover re-fetches archived bodies above the durable tip; those blocks come back through the same confirm.

Batch N+1 cannot commit, publish a tip, write strong bits, or append confirmed[] while N's barrier is in flight. The next class_c_commit joins first. A failed spawn still reaches the inline flush in spawn_tip_flush after the tip is published; success leaves TipFlush::Idle with the barrier already done, and a failure stores TipFlush::Failed. join_tip_flush puts Failed back after mem::take, so a one-block retry hits that error in lock_tip_commit before check_run_extends_tip can return the already-confirmed fk. A failed fsync stays failed until restart. failed_ibd_tip_flush_stops_commits_until_restart still fails the retry after the directory is writable again.

A kill during the overlap can leave the in-flight batch's spend annotations and filter or tweak puts above the durable tip, and the next batch's Class A bodies can be archived above that tip. Open still clamps an unsealed confirmed[] suffix with tip_seal, replays annotations down to the tip, and trims both indexes. The tip-hole cover is what fetches those archived bodies again. Seqsigwit prune still uses the height flush_tip returns for IBD, which is the pre-batch tip while that flush runs (ibd_seqsigwit_prune_follows_the_durable_tip expects 300 - keep, not the in-RAM tip).

dirty_try_clean still resets the low-water, then CASes the generation, and restores the mark if the CAS loses. TableFile::flush_between still clears needs_sync before persist_and_sync reads the length, and stores it again on error. ArrayTable::flush_dirty_between still holds flush_lock across the snapshot, so a bad-body header_txs flush cannot land an older image after the tip flush. The flush thread runs Store::flush_class_c_tip only. It does not take tip_commit, tip_flush, or the BIP34 mutex. Joiners take tip_commit and then tip_flush, and they do not hold a store flush lock while they join.

disconnect_and_drop_wait_for_the_ibd_tip_flush holds the helper and fails if disconnect or Drop returns early. ibd_commit_flushes_inline_when_its_thread_cannot_start reopens on the committed tip. set_during_flush_clean_is_flushed_next, write_during_flush_stays_pending, and concurrent_flushes_keep_the_newer_image still fail if their hooks' races come back. changelog.d/ibd-tip-flush-overlap.md and the Flush section of docs/crash-recovery.md still match this order; the concurrency row kept the helper thread and picked up master's Class A sentence.

Issue counts by severity

• bugs: 0
• suggestions: 0
• nits: 0