Skip to content

fix(spec): format timestamp and binary partition values like iceberg-java - #3352

Draft
JeonDaehong wants to merge 2 commits into
apache:mainfrom
JeonDaehong:fix-human-string-timestamp-binary
Draft

JeonDaehong wants to merge 2 commits into
apache:mainfrom
JeonDaehong:fix-human-string-timestamp-binary

Conversation

@JeonDaehong

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

What changes are included in this PR?

Datum::to_human_string, which partition_to_path uses through Transform::to_human_string, now matches iceberg-java for:

  • timestamp / timestamp_ns: yyyy-MM-ddTHH:mm:ss, with trailing zeros of the fraction trimmed (.120000 -> .12, no fraction for whole seconds)
  • timestamptz / timestamptz_ns: the same, followed by +00:00
  • binary / fixed: standard Base64 with padding

Display for Datum is unchanged, so this only affects partition paths and the partitions.<path> keys in the snapshot summary. Reads are not affected, since they take partition values from manifests. Tables already written by iceberg-rust with these partition types will get the new path format for new files.

Are these changes tested?

Yes.

  • New unit tests in transform.rs cover timestamps (micro and nano, with and without zone, whole seconds, trimmed fractions, pre-epoch values) and binary/fixed (empty, padding cases).
  • A partition.rs test checks a full path.
  • All expected values were produced by running iceberg-java's Transform.toHumanString / PartitionSpec.partitionToPath on main.

cargo test -p iceberg --lib (1845 passed), cargo clippy -p iceberg --all-targets and cargo fmt --check are clean.

AI Disclosure

I used Claude Code to help trace the Rust code path and draft the implementation and tests. I generated the expected values by running iceberg-java myself, and I reviewed the code and ran the tests and lints above.

@JeonDaehong

Copy link
Copy Markdown
Contributor Author

@laskoviymishka @kevinjqliu

Hello.
Could you take a look when you have time? This follows the same direction as #3022, aligning to_human_string with iceberg-java, this time for identity-partitioned timestamp and binary values.

Thank you

@JeonDaehong

Copy link
Copy Markdown
Contributor Author

Hello. @blackmwk
would you have time to take a look? This aligns timestamp/binary partition paths with iceberg-java, in the same area as #2773. CI is green.

@JeonDaehong
JeonDaehong force-pushed the fix-human-string-timestamp-binary branch from 7da837a to 2cee4b7 Compare October 15, 2026 11:42
@JeonDaehong

Copy link
Copy Markdown
Contributor Author

Rebased on #3358, which changed the same function: to_human_string now handles floats/doubles from #3358 and timestamps/binary from this PR.

@JeonDaehong
JeonDaehong marked this pull request as draft October 16, 2026 06:23
@JeonDaehong

Copy link
Copy Markdown
Contributor Author

Converted to draft to stay within the new limit of 3 open PRs (#3465). The change is still ready for review; I will mark it ready again when one of my other PRs lands.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Partition paths format timestamp and binary values differently from iceberg-java

1 participant