You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
truncate Local FS objects before complete replacement writes
clear stale .uncompressed-size sidecars after successful uncompressed replacements
cover Put and uncompressed StoreFile across shorter, longer, equal-length, and empty replacements
cover compressed-to-uncompressed replacement through both public write paths
Why
The Local FS backend opened existing files without O_TRUNC. Writing a shorter replacement started at offset zero but retained the previous suffix, unlike object-storage put semantics.
A compressed object also leaves an .uncompressed-size sidecar that Size() prefers. If the same object is later replaced through an uncompressed path, that sidecar must be removed or Size() reports the old value even when the replacement bytes are correct.
The truncation flag remains limited to getHandle(false), whose only callers are complete-object replacement paths. Sidecar cleanup happens only after a successful uncompressed copy. Read/range-read and compressed StoreFile behavior remain unchanged.
Validation
replacement matrix for Put and uncompressed StoreFile: shorter, longer, equal-length, and empty
compressed-to-uncompressed sidecar regression for both write paths
focused Local FS suite
race-enabled Local FS suite
all storage tests not requiring Docker/testcontainers
package compile-only test
golangci-lint run --new-from-rev=upstream/main ./... from packages/shared (0 issues)
The complete storage suite was also attempted; its Docker-backed Azure, GCS, and S3 integration tests cannot start locally because rootless Docker is unavailable.
Follow-up scenario verification found one additional Local FS replacement edge case and it is now included in b30f27fb9.
A compressed write creates an .uncompressed-size sidecar, and Size() prefers that value. Before this follow-up, replacing the same object through Put or uncompressed StoreFile fixed the bytes but left the old sidecar, so a 5-byte replacement could still report size 63. The new regression reproduced that on both public paths before the fix.
The uncompressed paths now remove the stale sidecar only after a successful copy. I also expanded both replacement regressions to cover shorter, longer, equal-length, and empty payloads.
Fresh verification after the update:
focused Local FS smoke suite: pass;
focused Local FS race suite: pass;
all non-Docker storage tests: pass;
package compile check: pass;
golangci-lint: 0 issues;
git diff check: pass.
The Docker-backed integration tests remain unavailable locally because rootless Docker is not installed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.uncompressed-sizesidecars after successful uncompressed replacementsPutand uncompressedStoreFileacross shorter, longer, equal-length, and empty replacementsWhy
The Local FS backend opened existing files without
O_TRUNC. Writing a shorter replacement started at offset zero but retained the previous suffix, unlike object-storage put semantics.A compressed object also leaves an
.uncompressed-sizesidecar thatSize()prefers. If the same object is later replaced through an uncompressed path, that sidecar must be removed orSize()reports the old value even when the replacement bytes are correct.The truncation flag remains limited to
getHandle(false), whose only callers are complete-object replacement paths. Sidecar cleanup happens only after a successful uncompressed copy. Read/range-read and compressedStoreFilebehavior remain unchanged.Validation
Putand uncompressedStoreFile: shorter, longer, equal-length, and emptygolangci-lint run --new-from-rev=upstream/main ./...frompackages/shared(0 issues)The complete storage suite was also attempted; its Docker-backed Azure, GCS, and S3 integration tests cannot start locally because rootless Docker is unavailable.
Closes #3642.