Skip to content

[AMORO-4375][Iceberg] Prevent automatic changes to unsupported V3 tables - #4376

Open
j1wonpark wants to merge 1 commit into
apache:masterfrom
j1wonpark:fix/iceberg-v3-maintenance-guard
Open

[AMORO-4375][Iceberg] Prevent automatic changes to unsupported V3 tables#4376
j1wonpark wants to merge 1 commit into
apache:masterfrom
j1wonpark:fix/iceberg-v3-maintenance-guard

Conversation

@j1wonpark

Copy link
Copy Markdown
Contributor

Why are the changes needed?

Amoro’s bundled Iceberg 1.8.1 writer can omit next-row-id when committing metadata for V3 tables created by newer writers, making them unreadable by Iceberg 1.9+.

The existing V3 check only disables Parquet row-group merging and does not prevent other automatic changes.

Close #4375.

Brief change log

  • Check format compatibility before optimizing evaluation, planning, and recovery.
  • Guard maintenance and metadata commits, including retries after an external V3 upgrade. Orphan cleanup checks compatibility before deleting files.
  • Preserve recovery during catalog lookup failures, disabled maintenance behavior, and read-only summary collection.
  • Record unsupported-version closure reasons and document the V1/V2 support boundary.

This patch prevents unsafe automatic changes; it does not add V3 support or repair affected metadata.

How was this patch tested?

  • Add positive and negative tests.
  • Run tests locally.

All 50 new test cases, including parameterized cases, passed, covering metadata preservation, supported versions, concurrent upgrades, retries, and recovery.

Related regression tests: 218 passed, 62 existing skips. After narrowing the orphan-cleanup wrapper scope, 63 targeted cases passed.

Full Maven validation passed before that final adjustment; targeted reactor style checks passed afterward. Separate Iceberg 1.9.2 verification confirmed reads and writes remain possible after maintenance is rejected.

The full repository test suite was not run. Screenshots are not applicable.

Documentation

  • Does this pull request introduce a new feature? No.
  • Updated docs/formats/iceberg.md with the compatibility boundary.

Signed-off-by: Jiwon Park <jiwonpark@apache.org>
@github-actions github-actions Bot added type:docs Improvements or additions to documentation module:ams-server Ams server module labels Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module:ams-server Ams server module type:docs Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Automatic maintenance can invalidate Iceberg V3 table metadata

1 participant