Enforce open-mode rules and add rekey_into_writer for restored stores - #45
Merged
Merged
Conversation
Each native backend (Tokio, io_uring, GCD, IOCP) kept its own in-process lock table, so distinct VFS instances rooted at the same directory could each believe they held the OS lock. GCD and IOCP also each carried their own bespoke fcntl/LockFileEx implementation. Replace the per-VFS tables with one process-wide table in oslock, keyed by the resolved (canonicalized parent + file name) lock path, so every instance over one directory contends correctly regardless of path spelling. A per-domain gate serializes acquiring the OS lock, and the single OS lock descriptor is now owned by the shared entry instead of each handle, so an early handle close can no longer drop the F_SETLK lock out from under a remaining in-process holder. GCD and IOCP now route through the same implementation instead of their own copies.
A snapshot directory and a directory `restore_from` fills copy `main.db` byte for byte, so they share the source's DEK and nonce space with it. Opening one Standalone previously proceeded anyway, letting independent writes on both directories repeat nonces under one key. Record a restore mode (STANDALONE, READ_ONLY, FOLLOWER) in the main.db header. A Standalone open now refuses any non-STANDALONE mode with RestoredNotPromoted. A restored directory can still open ReadOnly, or promote to Follower and track its source. `rekey_into_writer` forks a ReadOnly or Follower handle into a Standalone writer under a fresh identity: it rekeys the tree and quota catalogs and rewrites segments into a fork directory the open-time orphan scan skips, then adopts those segments into staging and publishes a fresh header once the fork takes the writer sentinel. Thread the KEK-changing-rekey resume path through open_with_mode via an optional counterpart KEK, and remove_if_present to make crash cleanup of scratch files idempotent.
farhan-syah
force-pushed
the
fix/observer-retry-scope
branch
from
September 27, 2026 21:14
fc2a5cb to
7badf9a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Db::open_existing_with_counterpart_keknow opens throughopen_with_mode. It holds the writer sentinel, reportsNotFoundfor a missing store, refuses a restored directory, and reads a page that fails authentication once instead of 4 times. The retry count is set where the pager is configured, so no open path can skip it. Only anObserverretries.snapshot_toandrestore_fromstamp both A/B header slotsREAD_ONLY.promote_to_followerand the newDb::open_followerrecordFOLLOWER. A Standalone open of either reportsRestoredNotPromoted.restore_mode,flags, and the retention policy from state instead of writing 0.Db::open_followerreopens a Follower directory and replays an interrupted apply journal.Db::rekey_into_writerforks aReadOnlyorFollowerhandle into an independent Standalone writer. It re-encrypts every page and segment under a freshfile_id, a freshkek_salt, and the supplied KEK intomain.db.fork, and renames that overmain.db. Realm, commit ids, keys, values, segment page ids, counters, and quotas carry over. Commit history starts empty. Fork segments wait inseg/.fork, which the orphan scan skips, and the first Standalone open adopts them into staging.src/vfs/oslock.rskeeps one process-wide table, keyed by resolved lock-file path, and one OS lock per file. Before, each VFS instance had its own table, and macOSF_SETLKnever conflicts within a process, so two handles on one directory could both take the writer sentinel. The GCD and IOCP backends drop their own lock copies and call the shared one.pager/header.rs(authenticate_slot,authenticate_slot_with_kek).vfs::remove_if_presentreplaces three copies.STAGING_DIRandstaging_pathreplace literal staging paths.Why
restore_modewas never written, soRestoredNotPromotednever fired. A restored copy shares its source'sfile_id,kek_salt, andmk_epoch: one key and one nonce space. Opened as a Standalone writer, it sealed pages with nonces its source also issues.open_existing_with_counterpart_kekbypassedopen_with_mode. A second writer could attach beside its handle, and it kept the Observer retry count.How to check it
cargo nextest run --all-features: 841 passed, 10 skipped.tests/open_mode_discipline.rs,tests/restored_store_modes.rs,tests/restored_store_fork.rs. They cover each open path, the restore-mode lifecycle, the fork's fresh salt, and a fork interrupted before and after the rename.cargo clippy --all-targets --all-features -- -D warnings,RUSTDOCFLAGS='-D warnings' cargo doc, thewasm32-unknown-unknown --features opfsandwasm32-wasip1checks, andcargo check --all-targetsforaarch64-apple-darwinandx86_64-pc-windows-gnuare clean.