Fix Playwright fork nightly 'tests BiDi' failed 30 consecutive nights — stale fork + missing Azure secrets, second dead nightly signal

[polylane]

Fixes: Playwright fork nightly 'tests BiDi' failed 30 consecutive nights — stale fork + missing Azure secrets, second dead nightly signal

The nightly 'tests BiDi' workflow on this fork fails every night in part because the Azure Login and report-upload steps run unconditionally on main-branch runs, and forked repositories do not inherit the AZURE_BLOB_REPORTS secrets the steps require, so the job reddens even when the bidi tests pass. This change skips those two steps unless the secrets are present, leaving the test result itself as the run's signal.

What caused this

Affected: int_01ace0dc7001iivrvgsb7n32 · severity low

What changed

• .github/workflows/tests_bidi.yml: added && secrets.AZURE_BLOB_REPORTS_CLIENT_ID != '' to the if guard of the Azure Login and Upload report.csv to Azure steps, so runs on forks without the upstream report secrets skip the upload path instead of failing on it.

Why it's safe

• On upstream microsoft/playwright, where the secrets exist, the added condition is true and both steps run exactly as before; the guard only changes behavior on runs without the secrets.
• The condition only gates the two Azure report steps; nothing in the test, build, or upload-to-artifact path changes.
• When the secrets are absent the steps skip instead of failing, which is the workflow's own intent for non-main forks: the branch guard github.ref == 'refs/heads/main' already exists and the upload steps are report publication, not part of the test signal.

Validation

• npx -y js-yaml .github/workflows/tests_bidi.yml: parses cleanly (exit 0); both Azure steps carry the new guard.
• node quote-balance and guard-count check: 2 guarded steps, quotes balanced.
• not run: npm run flint: workflow-only YAML change, no TS/JS or DEPS surfaces touched.

Root cause and scoping notes

Root cause

• Signal: nightly run of workflow 'tests BiDi' (296267544) concluded failure for 30 consecutive nights; the 09-12 run (34669889538) job log shows ##[error]Login failed with Error: Using auth-type: SERVICE_PRINCIPAL. Not all values are present. Ensure 'client-id' and 'tenant-id' are supplied. on both matrix legs.
• Mechanism: azure/login@v3 runs with if: ${{ !cancelled() && github.ref == 'refs/heads/main' }} and expands secrets.AZURE_BLOB_REPORTS_CLIENT_ID/TENANT_ID/SUBSCRIPTION_ID in its with block. GitHub does not propagate upstream secrets to forks, so the action fails on missing inputs; the following azcopy upload step fails after it.
• Root cause: the workflow is an unmodified upstream snapshot (byte-identical 3828 bytes at the fork's 4c6ee93 and upstream d1ead3e). Upstream leaves the steps unguarded because it has the secrets; the fork never can.
• Why this fixes the root rather than the symptom: the steps are permanently inexecutable on this fork, so the guard converts an unconditional step failure into the intended skip, matching the steps' own github.ref == 'refs/heads/main' intent (report upload only on upstream main). Verified verbatim error text from job 103489135979 and fork repo metadata (fork=true, pushed 08-21, 0 dependents).

Causal chain

• Signal (log template): ##[error]Login failed with Error: Using auth-type: SERVICE_PRINCIPAL. Not all values are present. Ensure 'client-id' and 'tenant-id' are supplied.
• Surfacing site: endformdev/playwright .github/workflows/tests_bidi.yml job test_bidi at endformdev/playwright:.github/workflows/tests_bidi.yml#Azure Login step (azure/login@v3)
• Mechanism: azure/login@v3 runs on every main-branch run (guard only !cancelled() && github.ref == 'refs/heads/main') and expands secrets.AZURE_BLOB_REPORTS_CLIENT_ID/TENANT_ID/SUBSCRIPTION_ID in its with block; forks do not inherit upstream secrets, so the action fails on missing inputs and the azcopy upload step fails after it.
• Producer: tests_bidi.yml workflow definition on the fork, instance endformdev/playwright fork, main branch, at endformdev/playwright:.github/workflows/tests_bidi.yml#Azure Login / Upload report.csv to Azure
• Trigger: Nightly cron (0 0 * * *) firing on fork main, frozen at 4c6ee93 since 08-21; 30 consecutive nightly failures 08-14..09-12
• What happens to the failed unit today: Scheduled run dropped with no retry; the Azure failure reddens the job on top of genuine test failures; no notify step in this workflow, so only Polylane's signal sees it.
• Cadence check: Unconditional: the secrets are permanently absent on the fork and the guard is true on every scheduled main run, matching 30/30 consecutive nightly failures.
• Blast radius: 0 other resource(s), 0 other tenant(s); data at risk: No user or product data at risk. Residual genuine test failures (15 assertions on the 09-12 firefox leg: cookie partitionKey, CSP bypass, HAR round-trip, goBack/goForward) are the stale-fork problem tracked as a follow-up on the issue timeline.
• Producer evidence:
  • Job log 103489135979 (09-12 run 34669889538): '##[error]Login failed with Error: Using auth-type: SERVICE_PRINCIPAL. Not all values are present. Ensure client-id and tenant-id are supplied.'
  • Workflow file is byte-identical to upstream at both fork HEAD 4c6ee93 and upstream main d1ead3e (3828 bytes); upstream leaves the steps unguarded because it has the secrets
  • Fork metadata: fork=true, parent=microsoft/playwright, last push 2026-08-21, 0 open PRs, 0 forks, 0 code dependents

Detection outcome

• Fingerprinted signal: the ##[error]Login failed with Error: Using auth-type: SERVICE_PRINCIPAL. Not all values are present. Ensure 'client-id' and 'tenant-id' are supplied. failure in the Azure Login step of the nightly 'tests BiDi' run (workflow 296267544, run 34669889538).
• This change stops that signal because the Azure Login step is skipped whenever AZURE_BLOB_REPORTS_CLIENT_ID is absent, which is permanently true on this fork (forks do not inherit upstream secrets). The step can no longer emit the login-failed error, and the subsequent Upload report.csv to Azure step is skipped with it.

This change is a declared stopgap: it contains the symptom while the producer's fix is tracked as a follow-up on the issue timeline.

Fix chosen

Chosen: cause (removes the mechanism that produces the failure): Guard the Azure Login and Upload report.csv steps with secrets.AZURE_BLOB_REPORTS_CLIENT_ID != '' so the permanently-missing secrets skip the report-upload path instead of failing the job.

Considered and not chosen:

• suppress (silences, downgrades, or reroutes the signal without changing what produces it): Disable the Azure steps entirely on the fork by removing them from tests_bidi.yml. Not chosen: Removing upstream workflow content would diverge the fork from upstream and break report upload if the fork is ever revived with secrets; a conditional guard preserves upstream behavior on upstream.
• loosen (raises a limit, threshold, or timeout, or widens a retry): Set the two Azure steps' guard to only github.ref == 'refs/heads/main' with no secrets check. Not chosen: That is the current state and it fails every night; it changes nothing.
• disable (turns a feature, guard, check, test, or telemetry off): Turn off the whole 'tests BiDi' scheduled workflow on the fork via cron configuration. Not chosen: The genuine bidi test failures are real signal (stale fork vs browser nightlies); disabling the workflow would hide them, and the fork's fate is a human decision tracked on the issue timeline, not something this change should preempt.

Outcome after fix

• Triggering input: a nightly scheduled run of tests_bidi.yml on endformdev/playwright@main, where AZURE_BLOB_REPORTS_CLIENT_ID is not set.
• New behavior at the failure point: the Azure Login and Upload report.csv to Azure steps evaluate their guard, find secrets.AZURE_BLOB_REPORTS_CLIENT_ID empty, and are skipped; the azcopy upload no longer runs and no step fails on missing credentials.
• What an external caller sees: the workflow run concludes based on the Run tests step alone, so a night where the bidi tests pass is green instead of red; a night where they fail is still red, with the Azure steps skipped above the test failure.
• Fate of the failed unit: previously the failed Azure step reddened the job on every run regardless of test results; now the run's conclusion reflects the actual test outcome, and report publication simply does not happen on the fork (it could never succeed there).

1 file changed (+2/-2)

• .github/workflows/tests_bidi.yml: modified, +2/-2

Repository conventions present in the repository: CLAUDE.md, .github/copilot-instructions.md, .claude/skills (playwright-dev, playwright-devops, playwright-test-results, playwright-triage).

Repository lint: npm run lint (declared in package.json) did not finish within its time budget, so its verdict is not known; run it before merging.

---

Generated by Polylane.

[github-actions]

Test results for "MCP"

1 failed
❌ [chrome] › mcp/http.spec.ts:450 › client should receive list roots request @mcp-macos-latest-chrome

8141 passed, 1361 skipped

---

Merge workflow run.

[github-actions]

Test results for "tests 1"

23 flaky

⚠️ [chromium-library] › library/global-fetch.spec.ts:293 › should return security details from response `@frozen-time-library-chromium-linux`
⚠️ [chromium-library] › library/har.spec.ts:639 › should have security details `@frozen-time-library-chromium-linux`
⚠️ [chromium-library] › library/global-fetch.spec.ts:293 › should return security details from response `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/har.spec.ts:639 › should have security details `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/video.spec.ts:736 › screencast › should work with video+trace `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/global-fetch.spec.ts:293 › should return security details from response `@realtime-time-library-chromium-linux`
⚠️ [chromium-library] › library/har.spec.ts:639 › should have security details `@realtime-time-library-chromium-linux`
⚠️ [chromium-library] › library/global-fetch.spec.ts:293 › should return security details from response `@chromium-ubuntu-22.04-node24`
⚠️ [chromium-library] › library/har.spec.ts:639 › should have security details `@chromium-ubuntu-22.04-node24`
⚠️ [chromium-library] › library/popup.spec.ts:260 › should not throw when click closes popup `@chromium-ubuntu-22.04-node24`
⚠️ [chromium-library] › library/global-fetch.spec.ts:293 › should return security details from response `@chromium-ubuntu-22.04-node20`
⚠️ [chromium-library] › library/har.spec.ts:639 › should have security details `@chromium-ubuntu-22.04-node20`
⚠️ [chromium-library] › library/video.spec.ts:736 › screencast › should work with video+trace `@chromium-ubuntu-22.04-node20`
⚠️ [chromium-library] › library/global-fetch.spec.ts:293 › should return security details from response `@chromium-ubuntu-22.04-node22`
⚠️ [chromium-library] › library/har.spec.ts:639 › should have security details `@chromium-ubuntu-22.04-node22`
⚠️ [chromium-library] › library/video.spec.ts:736 › screencast › should work with video+trace `@chromium-ubuntu-22.04-node22`
⚠️ [firefox-library] › library/global-fetch.spec.ts:293 › should return security details from response `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/har.spec.ts:639 › should have security details `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/trace-viewer-scrub.spec.ts:177 › should drag scrubber to select action `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/video.spec.ts:294 › screencast › should capture navigation `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-page] › page/page-event-request.spec.ts:181 › should return response body when Cross-Origin-Opener-Policy is set `@firefox-ubuntu-22.04-node20`
⚠️ [webkit-library] › library/global-fetch.spec.ts:293 › should return security details from response `@webkit-ubuntu-22.04-node20`
⚠️ [webkit-library] › library/har.spec.ts:639 › should have security details `@webkit-ubuntu-22.04-node20`

51251 passed, 1235 skipped

---

Merge workflow run.