Skip to content

Glue schema conversion should not fail for display-only Iceberg types #3375

Open

Description

@wirybeaver

Problem

GlueSchemaBuilder converts Iceberg types into Glue column type strings. These strings are informational for Iceberg tables: readers use Iceberg table metadata as the source of truth.

The current Rust converter returns FeatureUnsupported for some Iceberg types such as Unknown, Timestamptz, and TimestamptzNs. That can prevent otherwise valid Iceberg table operations merely because Glue cannot represent the type as a native processing type.

The Java IcebergToGlueConverter explicitly documents this conversion as display-only and always produces a string, while retaining Glue/Hive-compatible spellings for known types:
https://github.com/apache/iceberg/blob/main/aws/src/main/java/org/apache/iceberg/aws/glue/IcebergToGlueConverter.java#L294-L350

Proposed behavior

Refactor the Glue conversion so every Iceberg type has a non-failing display representation, while preserving existing Glue-compatible mappings such as long to bigint, time/UUID to string, and fixed to binary.

Raised from review discussion on #2933:
#2933 (comment)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions