[api][core] Reset Format Table partition locations on overwrite

[sundapeng]

Purpose

Follow-up to #9540. Every Format Table write commit currently reads the whole partition registry to find out whether it touches a partition registered at a custom location, and it rejects overwrite and truncate of such partitions. On a table with hundreds of thousands of partitions that is hundreds of REST calls per commit, for every writer.

This PR changes both sides:

• REST contract: partitionOptions[i].path = null resets a partition to its default location. It is accepted only with replaceStatistics=true and a partitionStatistics entry for the same spec, both applied in the same request. Omitting path keeps the stored location; non-null paths, other options and statistics-only requests behave as before. Additive statistics for a partition that already has a custom location are rejected. Servers without the reset reject the request, since a null option value already fails their validation.
• FormatTableCommit: appends look up only the partitions they wrote, static overwrites list only their prefix, and overwrite and truncate reset the target partitions to the default location together with replacement statistics instead of failing. External data is left untouched. Whole-table overwrite and truncate still list the table, which they need for directory cleanup.

Tests

• RESTApiJsonTest, MockRESTCatalogTest, CatalogFormatTablePartitionManagerTest
• FormatTableCommitTest, FormatTableCommitStatisticsTest, FormatTableCommitRegistryValidationTest
• node scripts/validate-rest-openapi.js

[JingsongLi]

Could we pass the concrete target partition path instead of using path: null as a reset operation?

The writer already knows its output directory. The REST server could derive and normalize the default partition path from the table metadata and partition spec, then remove the stored path override when the requested path equals that default. Whether the server stores this as NULL should be an implementation detail. The API semantics would be:

• Omit path: keep the current location.
• Provide path: specify the desired location.

For this PR, we could limit changes to existing locations to switching back to the default directory. Such a change should still require replacement statistics for the same partition and apply both atomically; an ordinary createPartitions(ignoreIfExists=true) should not silently relocate an existing partition.

This would also require recognizing the default path before the custom-location overlap/conflict checks. Passing a non-null path alone does not provide compatibility with older servers, so the updated server contract would still need to be explicit.

[sundapeng]

Makes sense — I'll switch to naming the default path. The preconditions you list are already what the PR enforces.

[JingsongLi]

Two path-normalization issues found while reviewing the explicit default-path contract.

[JingsongLi]

[P1] Preserve escaped partition directory names when normalizing default paths

FormatTableCommit now sends generated default partition paths through this code, but canonicalizeLocation() percent-decodes the Hive-escaped directory name. For a valid partition value dt=a%b, the generated directory is dt=a%25b; decoding leaves %, which causes the client to reject the path. Values such as a/b and a=b are also decoded into different directory names, so they no longer match the server's generated default path.

I reproduced an overwrite where the old file had already been deleted and the replacement published before this failure, leaving the operation failed and the old statistics unchanged. The same cases succeed with the baseline committer.

Please preserve the actual escaped directory identity throughout request normalization and default-path comparison, and add overwrite/truncate coverage for these partition values.

[JingsongLi]

[P2] Use table-location rules when recognizing a default partition path

This still calls canonicalizeCustomLocation(), which rejects locations without an authority for schemes other than file and hdfs. A supported table location such as viewfs:/warehouse/table therefore fails the basic round trip: generate its default directory for dt=2026, pass that exact path to isDefaultPartitionPath(), and the result is false.

The reference REST server's request preprocessing applies the same custom-location restriction. With a real ViewFS default mount and that server validation, I reproduced truncate deleting the old file before the reset request was rejected; the baseline succeeds.

Please use table-location canonicalization for default-path detection and request preprocessing, applying the stricter custom-location rules only after determining that a path is not the partition's own default directory.