Skip to content

test: skip flaky LLM checks in Maven resolve UI test - #1748

Merged
wenyt (wenytang-ms) merged 2 commits into
microsoft:mainfrom
wenytang-ms:wenytang-ms/skip-maven-resolve-llm-flakes
Sep 3, 2026
Merged

test: skip flaky LLM checks in Maven resolve UI test#1748
wenyt (wenytang-ms) merged 2 commits into
microsoft:mainfrom
wenytang-ms:wenytang-ms/skip-maven-resolve-llm-flakes

Conversation

@wenytang-ms

@wenytang-ms wenyt (wenytang-ms) commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • skip LLM verification for two Maven resolve steps that already have deterministic assertions
  • establish an exact 0 -> 1 diagnostics transition around the unresolved Gson insertion
  • verify the inserted Java field on disk and require the pom.xml editor tab to be visible

Context

The vscode-java 1.56.0 candidate validation failed twice on Ubuntu at different LLM-only checks even though the deterministic behavior completed successfully:
https://github.com/microsoft/vscode-java-pack/actions/runs/33704157866

Validation

  • npx -y @vscjava/vscode-autotest validate test-plans\java-maven-resolve-type.yaml
  • java-maven-resolve-type passed on Windows, Ubuntu, and macOS in PR CI

@wenytang-ms wenyt (wenytang-ms) changed the title Skip flaky LLM checks in Maven resolve UI test test: skip flaky LLM checks in Maven resolve UI test Sep 3, 2026
@wenytang-ms
wenyt (wenytang-ms) merged commit afb6365 into microsoft:main Sep 3, 2026
108 of 110 checks passed
@wenytang-ms
wenyt (wenytang-ms) deleted the wenytang-ms/skip-maven-resolve-llm-flakes branch September 3, 2026 02:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants