Skip to content

[main] Update MSTest and Microsoft.Testing.Platform to 26459.12 - #11138

Merged
Amaury Levé (Evangelink) merged 5 commits into
mainfrom
dependabot/nuget/eng/multi-07ed09bcaa
Sep 10, 2026
Merged

Amaury Levé (Evangelink) merged 5 commits into
mainfrom
dependabot/nuget/eng/multi-07ed09bcaa

Conversation

@dependabot

@dependabot dependabot Bot commented on behalf of github Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

Updates the released test dependencies to the latest aligned build from microsoft/testfx:

  • MSTest.Analyzers, MSTest.TestAdapter, and MSTest.TestFramework: 4.5.0-preview.26459.12
  • Microsoft.Testing.Platform.MSBuild: 2.5.0-preview.26459.12

eng/Versions.props, Directory.Packages.props, and eng/Version.Details.xml resolve these packages consistently. Both product builds originate from commit 0989da8c698e9a960088b698d668e54413e2199a.

Also keeps IDE0300 at suggestion severity for test sources because the new assertion overloads otherwise promote existing collection-initialization style suggestions to build-breaking warnings.

Supersedes #11166 and #11167.

…view.26458.17

---
updated-dependencies:
- dependency-name: MSTest.TestFramework
  dependency-version: 4.5.0-preview.26458.16
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: MSTest.TestFramework
  dependency-version: 4.5.0-preview.26458.17
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot dependabot Bot added the dependencies Updates to dependency manifests. label Sep 9, 2026
Copilot AI balanced review requested due to automatic review settings September 9, 2026 07:10
@dependabot dependabot Bot added the dependencies Updates to dependency manifests. label Sep 9, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The corresponding dependency version and SHA in eng/Version.Details.xml remain stale.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Updates the shared MSTest dependency version to the latest preview build.

Changes:

  • Bumps MSTestVersion from 4.5.0-preview.26457.5 to 4.5.0-preview.26458.17.
File summaries
File Description
eng/Versions.props Updates the shared MSTest package version.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 1
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread eng/Versions.props Outdated

Copilot AI commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Pipeline Test Triage — Final Resolution

Analysis: Final (completed build)
Azure build: [1588585]((dev.azure.com/redacted), source refs/pull/11138/merge

Finding: not a test-product defect — root cause is an upstream compilation/build failure, out of this agent's scope.

All 3 candidate failures in the evidence are MSTest.Acceptance.IntegrationTests acceptance tests (PackedMSTestTestAdapter_UwpPropsRegisterMSTestBuilderHook, PackedMSTestTestAdapter_ContainsRequiredWindowsApplicationModelAssets, PackedMSTestTestFramework_ContainsRequiredWindowsApplicationModelAssets) on the Windows application-model acceptance leg. Every one throws the same signature:

System.InvalidOperationException: Was expecting to find a single NuGet package named 'MSTest.TestFramework.' in 'D:\a\_work\1\s\artifacts\packages\Release\Shipping', but found 0.

That means AcceptanceTestBase could not find the packed MSTest.TestFramework NuGet at all — the Shipping packages directory was empty. Correlating with timeline.compact.json, the pipeline's Build task failed on this run with 29 error MSTEST0084 diagnostics ("Platform compatibility attributes ... not represented by an equivalent [OSCondition] attribute") across Microsoft.Testing.Platform.UnitTests files, so packing never produced the Shipping NuGets and every downstream acceptance test that depends on them failed as a cascade, not because of a genuine product/test regression.

This PR only bumps MSTest.TestFramework/analyzer versions in eng/Versions.props; the newer analyzer version most likely activated or tightened rule MSTEST0084, surfacing pre-existing gaps in the repository's own unit-test files. That is a build/compilation failure, which belongs to Build Failure Analysis, not test triage.

History: history.json (12 prior builds on main, incomplete: true) shows the three candidate test names normally passed or skipped (skipped on non-Windows legs, as expected). Only one unrelated prior build failed for a different reason. No prior occurrence of this specific "package not found" signature was found in the bounded history, but the incomplete flag means an earlier occurrence elsewhere cannot be ruled out.

Conclusion: No test-product defect, flaky test, or duration regression is present — the failing tests are a downstream symptom of the build-time MSTEST0084 compilation errors. No issue is being created by this test-triage workflow; recommend addressing the root MSTEST0084 violations (or the analyzer-version bump interaction) via the build-failure path, then re-running this PR's pipeline.

🤖 Automated content by GitHub Copilot. Generated by the Pipeline Test Triage workflow. · auto · 67.2 AIC · ⌖ 1.77 AIC · ⊞ 9.3K · [◷]( · ◷)

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Commit pushed: 94c6ea8

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · auto · 200.5 AIC · ⌖ 8.31 AIC · ⊞ 13.4K · ◷

Copilot AI review requested due to automatic review settings September 9, 2026 07:52
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Build failure analysis

Root cause: bumping MSTestVersion in eng/Versions.props (4.5.0-preview.26457.5 → 4.5.0-preview.26458.17) tightened nullable-reference-type annotations on Assert.Contains<T, TCollection> / Assert.DoesNotContain<T>, which broke two pre-existing test call sites that were previously accepted with nullable arguments. These errors reproduced identically on Linux Release, MacOS Release, Windows Debug, Windows Release (4 of 6 legs); the two application-model_acceptance legs on Windows compiled cleanly for their own scope, so this is a genuine, cross-platform build failure — not a flaky/non-build issue.

1. CS8604 — possible null reference argument

test/IntegrationTests/Microsoft.Testing.Platform.Acceptance.IntegrationTests/DotnetTestPipe/DotnetTestPipeArtifactPostProcessingTests.cs:257

Assert.DoesNotContain(unrelatedPath, artifacts[0].InputArtifactPaths);

InputArtifactPaths is string[]?. Although line 253 has an Assert.IsNotNull(artifacts[0].InputArtifactPaths) guard, the compiler doesn't narrow the re-evaluated member access at line 257 (it's a new property access, not the same local). Fix: add the null-forgiving operator, since nullability was already asserted two lines above:

-            Assert.DoesNotContain(unrelatedPath, artifacts[0].InputArtifactPaths);
+            Assert.DoesNotContain(unrelatedPath, artifacts[0].InputArtifactPaths!);

2. CS8631 — generic constraint nullability mismatch

test/UnitTests/Microsoft.Testing.Extensions.UnitTests/CtrfReportMergerTests.cs:54-55

List<string?> names = [.. testArray.Select(t => (string?)t!["name"])];
Assert.Contains("TestA", names);
Assert.Contains("TestC", names);

Assert.Contains<T, TCollection> now constrains TCollection : IEnumerable<T> with non-nullable T (inferred string), but names is List<string?>. The "name" JSON values are guaranteed non-null in this test's own fixture data. Fix: change the projection to non-nullable string:

-        List<string?> names = [.. testArray.Select(t => (string?)t!["name"])];
+        List<string> names = [.. testArray.Select(t => (string)t!["name"]!)];

Both fixes are mechanical, provable purely from the compiler errors, and confined to test/ files never touched by this PR's single-line Versions.props diff — so I pushed a follow-up commit applying them directly to the PR branch. Please review before merging.

Secondary noise (not addressed)

Many IDE0300/IDE0004 style-analyzer diagnostics (elevated to build errors) also appear across several files in test/UnitTests/Microsoft.Testing.Extensions.UnitTests/ (e.g. AzureDevOpsLivePublishingTests.cs, TrxArtifactPostProcessorTests.cs, JUnitReportMergerTests.cs, HtmlArtifactPostProcessorTests.cs, GitHubActionsSummaryReporterTests.cs, CtrfArtifactPostProcessorTests.cs). These look like pre-existing collection-initializer/cast-simplification suggestions rather than something newly introduced by the MSTest bump, and are too numerous/dispersed for a single mechanical fix — flagging for awareness but not fixed here.

Build overview (Linux Release leg)
Build: FAILED
Duration: 371.8s
MSBuild: 18.11.0-1.26425.121+e20a5af9c
Projects: 51  Errors: 190  Warnings: 1

Failed projects:
  Build.proj, NonWindowsTests.slnf,
  Microsoft.Testing.Platform.Acceptance.IntegrationTests.csproj,
  Microsoft.Testing.Extensions.UnitTests.csproj,
  Microsoft.Testing.Platform.UnitTests.csproj,
  Microsoft.Testing.Platform.ServerMode.Client.Sources.UnitTests.csproj

Same CS8604/CS8631 root causes recur in MacOS Release, Windows Debug, and Windows Release legs (Windows application-model_acceptance legs ×2 had no build errors).

Analyzed at commit b2183925f2f50394e3eca24b802b65a9ae538cc5 — workflow run 34324699625.

🤖 Automated content by GitHub Copilot. Generated by the Build Failure Analysis workflow. · auto · 200.5 AIC · ⌖ 8.31 AIC · ⊞ 13.4K · [◷]( · ◷)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The version bump and associated nullability adjustments are consistent and contain no unresolved issues.

Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Copilot AI commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

One or more custom setup steps configured for this repository failed during this Copilot code review run:

Build the repository

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 9, 2026 19:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The focused dependency update and compatibility adjustments preserve existing test behavior without unresolved issues.

Review details
  • Files reviewed: 6/6 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Copilot AI review requested due to automatic review settings September 9, 2026 20:08
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

🧪 Expert test review — PR #11138

No new or modified test methods were identified in the changed regions
of this PR. The diff only touches eng/Versions.props (a dependency
version bump) and test/.editorconfig (an analyzer severity rule
addition) — no test source files were changed. Nothing to review.

Re-run with /review-tests.

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 41.3 AIC · ⌖ 2.37 AIC · ⊞ 15.9K · [◷]( · ◷)

@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Dependency-flow metadata remains at .26458.14, and the PR description does not match the actual update.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (1)

eng/Versions.props:14

  • The dependency-flow metadata is now stale: eng/Version.Details.xml:20-26 still records both MSTest and Microsoft.Testing.Platform at .26458.14 with the old source SHA. The preceding Maestro update (#11125) changed Version.Details.xml and these properties together, so please update both dependency entries and their SHA to .17 (preferably through Maestro) to keep future dependency-flow operations accurate.
    <MicrosoftTestingPlatformVersion>2.5.0-preview.26458.17</MicrosoftTestingPlatformVersion>
  • Files reviewed: 2/2 changed files
  • Comments generated: 1
  • Review effort level: Balanced

Comment thread eng/Versions.props Outdated
…multi-07ed09bcaa

# Conflicts:
#	eng/Versions.props
Copilot AI review requested due to automatic review settings September 10, 2026 07:59
@Evangelink Amaury Levé (Evangelink) changed the title [main] Bump MSTest.TestFramework to 4.5.0-preview.26458.16, 4.5.0-preview.26458.17 [main] Update MSTest and Microsoft.Testing.Platform to 26459.12 Sep 10, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🧵 Parallel-safety audit — PR #11138

This PR only updates dependency versions (eng/Version.Details.xml, eng/Versions.props) and adds an unrelated .editorconfig severity tweak (IDE0300). No test files, source files under src/, or parallelization-configuration files (.runsettings, testconfig.json, .csproj/.props/.targets, or [assembly: Parallelize]/[DoNotParallelize] attributes) were changed.

Nothing audited here touches process-global state, shared filesystem paths, or [ResourceLock] / [DoNotParallelize] declarations. Nothing to flag for parallel-safety.

Re-run with /parallel-audit.

🤖 Automated content by GitHub Copilot. Generated by the Parallel-safety audit on PR (on open / sync) workflow. · auto · 32.5 AIC · ⌖ 2.91 AIC · ⊞ 23.9K · [◷]( · ◷)

@github-actions

Copy link
Copy Markdown
Contributor

🧪 Expert test review — PR #11138

No new or modified test methods were identified in the changed regions
of this PR. Nothing to review.

Re-run with /review-tests.

🤖 Automated content by GitHub Copilot. Generated by the Test Reviewer on PR (on open / sync) workflow. · auto · 36.9 AIC · ⌖ 2.31 AIC · ⊞ 15.9K · [◷]( · ◷)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The versions, package references, and dependency provenance are consistent, with no unresolved issues found.

Review details
  • Files reviewed: 3/3 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@Evangelink Amaury Levé (Evangelink) added the state/needs-review Awaiting review from the team. label Sep 10, 2026
@Evangelink
Amaury Levé (Evangelink) merged commit 2e3666b into main Sep 10, 2026
37 checks passed
@Evangelink
Amaury Levé (Evangelink) deleted the dependabot/nuget/eng/multi-07ed09bcaa branch September 10, 2026 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Updates to dependency manifests. state/needs-review Awaiting review from the team.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants