You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
JSON round-trips reject valid float partition values such as 123.456 because of an absolute f32::EPSILON check. NaN and ±Infinity silently become null for both float and double partitions, allowing incorrect metadata to be committed.
To Reproduce
Take test_insert_into_partitioned from datafusion-iceberg PR #54 and run it against the PR’s base commit, which still uses the JSON helpers.
Use 100.1 for the finite float partition value. The insert fails with Unable to convert raw literal (double) fail convert to type float.
Change it to 1.5, keeping the NaN, Infinity, and -Infinity rows. The insert succeeds, but inspecting the committed manifest shows null for those three partition values. File and row counts still look correct.
Repeat with a double partition column: non-finite values also silently become null.
Expected behavior
Preserve valid partition values. Reject unsupported values explicitly instead of silently converting them to null.
Activity