Repository navigation
[test] AzureFileSystemPluginTest spends ~10 min retrying DNS lookups #4637
Description
Search before asking
- I searched in the issues and found nothing similar.
Description
testCreateWithAccountKey and testCreateWithoutAccountKey call plugin.create() on abfs://container@account.dfs.core.windows.net/. That host does not exist (NXDOMAIN). The ABFS client then retries with exponential backoff (30 attempts, up to 25 s apart), which takes ~632 s on CI and sits on the core critical path. account is also a real, registrable storage-account name.
Fix:set fs.azure.io.retry.max.retries=0, or use a .invalid host. This should decrease the test time signnifically.
Willingness to contribute
- I'm willing to submit a PR!
Activity
dstolis commentedon Oct 14, 2026
@polyzos can i take this one?
polyzos commentedon Oct 14, 2026
@dstolis assigned to you
dstolis commentedon Oct 14, 2026
@polyzos I reproduced this locally (macOS, JDK 17, hadoop-azure 3.4.3).
AzureFileSystemPluginTestbefore and after the fix:Observations:
testCreateWithoutAccountKey.testCreateWithAccountKeyis fast onlybecause
"some-key"is not valid base64, so ABFS rejects it duringinitialize()before making anynetwork call. The exception is swallowed, so
create()is never really exercised.AbfsFileSystemBehaviorITCaseandAbfsFileSystemDelegationTokenITCaseuse a similar fake host andstay fast because they set
fs.azure.account.hns.enabled=false.Fix: #4640