[AMORO-4375][Iceberg] Prevent automatic changes to unsupported V3 tables - #4376
Open
j1wonpark wants to merge 1 commit into
Open
[AMORO-4375][Iceberg] Prevent automatic changes to unsupported V3 tables#4376j1wonpark wants to merge 1 commit into
j1wonpark wants to merge 1 commit into
Conversation
Signed-off-by: Jiwon Park <jiwonpark@apache.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are the changes needed?
Amoro’s bundled Iceberg 1.8.1 writer can omit
next-row-idwhen committing metadata for V3 tables created by newer writers, making them unreadable by Iceberg 1.9+.The existing V3 check only disables Parquet row-group merging and does not prevent other automatic changes.
Close #4375.
Brief change log
This patch prevents unsafe automatic changes; it does not add V3 support or repair affected metadata.
How was this patch tested?
All 50 new test cases, including parameterized cases, passed, covering metadata preservation, supported versions, concurrent upgrades, retries, and recovery.
Related regression tests: 218 passed, 62 existing skips. After narrowing the orphan-cleanup wrapper scope, 63 targeted cases passed.
Full Maven validation passed before that final adjustment; targeted reactor style checks passed afterward. Separate Iceberg 1.9.2 verification confirmed reads and writes remain possible after maintenance is rejected.
The full repository test suite was not run. Screenshots are not applicable.
Documentation
docs/formats/iceberg.mdwith the compatibility boundary.