Use avro for datafiles collection from write-exec

[hareshkh]

Replace JSON with Avro for data-file metadata passed from IcebergWriteExec to IcebergCommitExec. The JSON path could reject valid float partition values or silently commit NaN and ±Infinity as null for float and double partitions.

Each write task groups its files into one Avro container to avoid repeating the schema. Bump iceberg-rust to apache/iceberg-rust@5b82ab99a to support partitions with repeated fixed-width types.

Upstream JSON helper issue: apache/iceberg-rust#3409

[xanderbailey]

@mbutrovich interested to know your thoughts here

[mbutrovich]

Thanks @hareshkh. Passing the data files to the commit in the same Avro encoding that the manifest uses makes sense to me.

The JSON helpers this PR stops using are still public in iceberg-rust, and the bug is still there. On the base commit, the new integration test fails at the insert with Unable to convert raw literal (double) fail convert to type float. That comes from RawLiteral, which rejects narrowing an f64 to an f32 when the difference exceeds f32::EPSILON. That is an absolute bound, and 123.456 exceeds it. With a value that f32 holds exactly, such as 1.5, the insert succeeds, but the manifest records the NaN, Infinity, and -Infinity partition values as null. The same happens for a double partition column. As I read it, that part comes from serde_json writing non-finite floats as null in serialize_data_file_to_json. Could you file an issue on iceberg-rust for these helpers and link it from the description? Could the description also say that the old path committed wrong partition values without an error? Today it reads as if the old path only failed.

[hareshkh]

Thanks for the detailed review @mbutrovich. iceberg-rust json helpers bug report - apache/iceberg-rust#3409

[hareshkh]

@mbutrovich / @xanderbailey : The PR should be ready for another pass

[mbutrovich]

LGTM, thanks @hareshkh!