Skip to content

test(ci): pin WASM_BINDGEN_TEST_TIMEOUT to 70 in ci_wiring - #456

Merged
rubenhensen merged 2 commits into
mainfrom
dobby/pin-wasm-bindgen-test-timeout-454
Sep 24, 2026
Merged

rubenhensen merged 2 commits into
mainfrom
dobby/pin-wasm-bindgen-test-timeout-454

Conversation

@dobby-coder

@dobby-coder dobby-coder Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Part of #454, graduated from #453 and #247.

Read this first: one check is red on purpose, and I cannot turn it green

Test workspace (pg-core) fails, on the one assertion this PR adds:
the_wasm_browser_timeout_is_pinned_above_safaris_observed_tail. It fails
because .github/workflows/build.yml still sets WASM_BINDGEN_TEST_TIMEOUT: 30
while the assertion demands 70.

dobby-coder has no workflows: write, so the half of this change that lives in
build.yml cannot be pushed from here. That two-halves split is the thing
pg-core/tests/ci_wiring.rs was written to catch, and its file header spells it
out: the code and tests go in a PR, the workflow YAML is applied by hand, and
twice before, nothing noticed the second half never arrived. This PR is an
instance of that split, not an exception to it. Apply the patch below to this
branch and the check goes green; nothing else about the PR changes.

Test workspace (pg-cli), Test workspace (pg-pkg) and Test workspace (cryptify) also show red. They are not separate failures. All three ended with
conclusion cancelled, killed by the matrix's fail-fast when the pg-core job
failed, and their logs contain no failing test: pg-cli got through its 7 tests
green before the cancel reached it. Every other check passes, including all
three browser jobs.

What this does

test-wasm-browsers sets WASM_BINDGEN_TEST_TIMEOUT: 30. #453 established that
value is a timer over the whole wasm-pack test run, not per test, and that it
clips Safari's observed tail (up to 30.9s across 22 green runs) rather than
bounding a single slow test. This PR pins the correct value, 70 (~2x that tail),
as an assertion in pg-core/tests/ci_wiring.rs, beside the #432-added
assertions beneath it.

The gate is the literal number rather than a passing job on purpose. A timeout
planted near the suite's measured duration fails only the unlucky runs, so a
green browser job proves nothing about the margin left. Only a check on the
literal value catches a later edit drifting it back toward 30.

Workflow patch, for a maintainer to apply

This is what needs to land on dobby/pin-wasm-bindgen-test-timeout-454:

--- a/.github/workflows/build.yml
+++ b/.github/workflows/build.yml
@@ -98,12 +98,12 @@ jobs:
     # silence each (#416).
     timeout-minutes: 12
     env:
-      # ~6x the slowest single test (~5s). This is a per-test timer running
-      # inside the browser page, so it does NOT bound a wedged safaridriver
-      # session -- that is what `timeout-minutes` above is for. It was 120,
-      # which is both 24x the slowest test and, at 16 tests, a 32-minute
-      # worst case that the job budget could never reach.
-      WASM_BINDGEN_TEST_TIMEOUT: 30
+      # A timer over the whole `wasm-pack test` run, not per test: it starts
+      # after the page navigates and waits, in one clock, for `test result: `
+      # to appear across all 16 tests (#453). Safari's green runs take up to
+      # ~31s; 70 is ~2x that. It does NOT bound a wedged safaridriver session
+      # -- that is what `timeout-minutes` above is for.
+      WASM_BINDGEN_TEST_TIMEOUT: 70
     steps:
       - uses: actions/checkout@v4

The comment changes with the value. The old one describes a per-test timer, which
is the reading #453 disproved, so leaving it in place beside a raised number would
keep the wrong model on file.

Applying it needs the workflow token scope (gh auth refresh -s workflow).
Checking the branch out and editing line 106 works. So does this, which needs no
checkout, replaces that exact block and nothing else, and stops with expected 1 match, found 0 rather than writing anything if the file has drifted:

BR=dobby/pin-wasm-bindgen-test-timeout-454
REPO=encryption4all/postguard
F=.github/workflows/build.yml

python3 - "$REPO" "$BR" "$F" <<'PY' > /tmp/build.yml.b64
import base64, subprocess, sys
repo, br, f = sys.argv[1], sys.argv[2], sys.argv[3]
cur = subprocess.run(["gh", "api", f"repos/{repo}/contents/{f}?ref={br}", "--jq", ".content"],
                     capture_output=True, text=True, check=True).stdout
s = base64.b64decode(cur).decode()
old = """    env:
      # ~6x the slowest single test (~5s). This is a per-test timer running
      # inside the browser page, so it does NOT bound a wedged safaridriver
      # session -- that is what `timeout-minutes` above is for. It was 120,
      # which is both 24x the slowest test and, at 16 tests, a 32-minute
      # worst case that the job budget could never reach.
      WASM_BINDGEN_TEST_TIMEOUT: 30
"""
new = """    env:
      # A timer over the whole `wasm-pack test` run, not per test: it starts
      # after the page navigates and waits, in one clock, for `test result: `
      # to appear across all 16 tests (#453). Safari's green runs take up to
      # ~31s; 70 is ~2x that. It does NOT bound a wedged safaridriver session
      # -- that is what `timeout-minutes` above is for.
      WASM_BINDGEN_TEST_TIMEOUT: 70
"""
assert s.count(old) == 1, f"expected 1 match, found {s.count(old)}"
print(base64.b64encode(s.replace(old, new).encode()).decode())
PY

gh api -X PUT "repos/$REPO/contents/$F" \
  -f branch="$BR" \
  -f message="ci: raise WASM_BINDGEN_TEST_TIMEOUT to 70 (#454)" \
  -f sha="$(gh api "repos/$REPO/contents/$F?ref=$BR" --jq .sha)" \
  -f content="$(cat /tmp/build.yml.b64)"

(Also posted as a comment on #454.)

Verification

Re-run today against fd015e7, on top of the earlier round:

  1. The command above was run here as far as the base64 it produces, stopping
    short of the PUT I have no scope for. Its output decodes to a build.yml
    that differs from the branch's in exactly the six lines of the diff.
  2. With that file in place, cargo test --manifest-path pg-core/Cargo.toml --features test,rust,stream --test ci_wiring is 20 passed, 0 failed. The
    value alone (30 to 70, comment untouched) also passes, so the assertion
    reads the value and is not coupled to the comment text.
  3. Reverted, the same suite is 19 passed, 1 failed, on
    the_wasm_browser_timeout_is_pinned_above_safaris_observed_tail, with
    is 30, not 70. Values 30 and 69 both fail. git status is clean.
  4. On CI at fd015e7: pg-core's lib tests are 73/73 green and ci_wiring is
    19/20, the one failure being this assertion. Format, clippy, semver, wire
    compat and all three browser jobs pass.
  5. git diff --name-only origin/main...HEAD -- .github/workflows/ prints
    nothing. The only file this branch touches is pg-core/tests/ci_wiring.rs.

Nothing else in build.yml changes: timeout-minutes, the matrix and the run
command are untouched.

🤖 Generated with Claude Code

#453 established WASM_BINDGEN_TEST_TIMEOUT is a whole-run timer, not a
per-test one, and that 30 (set by #432) clips Safari's observed tail
(up to 30.9s) rather than the slowest single test. This assertion pins
the value at 70 -- ~2x that tail -- so a later edit cannot quietly
lower it back toward 30 behind a green run that proves nothing about
the margin left.

Red on this branch: dobby-coder has no workflows: write, so the
build.yml patch that raises the value and corrects its comment is
posted on #454 for a maintainer to apply.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

@dobby-coder dobby-coder Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rule check: 5 rules in force (atomic-commits, code-comments, draft-pull-requests, no-summary-issues, test-suite-before-submit) — all comply. Single well-described commit; the new test's doc-comment density matches its immediate neighbour (the_wasm_browser_job_has_an_honest_timeout_and_no_retry_wrapper) in both style and length; PR is already a draft; no summary/report issue was created (the patch was posted as a comment on the existing #454, not a new issue); the full ci_wiring suite was run before submission and the one failure is the documented, expected one.

Combined with the prior review pass: build is clean, cargo fmt --check and clippy pass, and the test suite runs 19/20 with the single expected failure (the_wasm_browser_timeout_is_pinned_above_safaris_observed_tail, red only because build.yml's WASM_BINDGEN_TEST_TIMEOUT is still 30 and this bot has no workflows: write to apply the linked patch). No code bugs found; the line-search/strip/trim logic is correct and matches the real current workflow value.

No blocking findings. Verdict: approve — a maintainer still needs to apply the build.yml patch posted on #454 to turn the one red check green. (Posted as COMMENT rather than APPROVE: GitHub rejects self-approval since this PR was opened by the same dobby-coder bot identity — the verdict itself is unaffected.)

@dobby-coder
dobby-coder Bot marked this pull request as ready for review September 23, 2026 12:16
Applies the workflow half of #456, which dobby-coder cannot push.
See #453 for the measurement.
@rubenhensen
rubenhensen merged commit bfd65b7 into main Sep 24, 2026
41 checks passed
@github-actions github-actions Bot mentioned this pull request Sep 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant