Skip to content

Add - ClaudeGrill Playwright QA suite - #110

Merged
iamprazol merged 4 commits into
masterfrom
add/claudegrill-qa-setup
Sep 24, 2026
Merged

iamprazol merged 4 commits into
masterfrom
add/claudegrill-qa-setup

Conversation

@iamprazol

Copy link
Copy Markdown
Contributor

Adds a Playwright e2e suite for Flash Free, plus the ClaudeGrill QA setup.

Comment @claudegrill suite to run the suite.

🤖 Generated with Claude Code

Playwright e2e suite for Flash Free: @fresh specs for verified theme
behaviour (runs on a bare Playground install and the demo site), @demo
specs for the imported demo and WooCommerce, and quarantined fixme
scenarios for open flash-pro issues #30, #84, #90 and #92.

Also adds the claudegrill manifest and knowledge file, the QA workflow,
.distignore, Playwright ignores, QA exclusions in the Grunt and gulp zip
tasks, and an .npmrc for the existing stylelint peer conflict.
@github-actions

Copy link
Copy Markdown

QA suite — passed ✅

All 19 checks passed.

14 passed · 0 failed · 5 skipped · 0 flaky · 32s

Automated check — no AI involved. It runs the tests in this branch.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Critical suite-manifest issues prevent reliable execution, with additional compatibility and test-isolation issues unresolved.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 2 High severity · 3 Medium severity

Open (5)
What changed in this PR

Adds a Playwright E2E QA suite for Flash Free and ClaudeGrill, covering fresh, demo, and WooCommerce environments.

Changes:

  • Adds frontend, WooCommerce, and quarantined regression specs.
  • Adds shared fixtures, authentication, helpers, and Playwright configuration.
  • Adds ClaudeGrill metadata, documentation, dependency setup, and distribution exclusions.
File Reviewed changes
tests/​e2e/​utils/​page.ts Navigation and preloader helpers.
tests/​e2e/​utils/​login.ts Admin session utilities.
tests/​e2e/​specs/​woocommerce/​header-cart.spec.ts Header cart coverage; cleanup must preserve existing quantities and run in finally (moderate, 3 and 1 votes).
tests/​e2e/​specs/​single-post/​single-post.spec.ts Single-post E2E coverage.
tests/​e2e/​specs/​search/​search-results.spec.ts Search-results coverage.
tests/​e2e/​specs/​not-found/​not-found.spec.ts 404-page coverage.
tests/​e2e/​specs/​mobile-menu/​mobile-menu-toggle.spec.ts Mobile-menu coverage.
tests/​e2e/​specs/​homepage/​front-page-health.spec.ts Homepage health coverage.
tests/​e2e/​specs/​header/​header-search.spec.ts Header-search coverage.
tests/​e2e/​specs/​footer/​scroll-to-top.spec.ts Scroll-to-top coverage.
tests/​e2e/​specs/​demo-homepage/​demo-front-page.spec.ts Demo homepage coverage; does not capture console.error (moderate, 2 votes).
tests/​e2e/​specs/​blog/​blog-index.spec.ts Blog-index coverage.
tests/​e2e/​fixtures.ts Anonymous visitor fixtures.
tests/​e2e/​auth.setup.ts Shared admin authentication setup.
playwright.config.ts Playwright projects and reporting configuration.
package.json Playwright dependency and test script; declared Node 8 support conflicts with the new tooling (moderate, 1 vote).
gulpfile.js Excludes QA files from ZIP output.
Gruntfile.js Excludes QA files from distribution archives.
.themegrill-qa/​suite.json ClaudeGrill manifest; browser provisioning and environment-variable mismatches are critical (1 vote each), and changed-only scope can skip suite changes (moderate, 2 votes).
.themegrill-qa/​knowledge.md QA knowledge base.
.themegrill-qa/​docs/​flash.md Documentation audit notes.
.npmrc Peer-dependency installation workaround.
.gitignore Ignores QA credentials and artifacts.
.distignore Excludes QA files from distributions.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .themegrill-qa/suite.json
Comment thread .themegrill-qa/suite.json
Comment thread .themegrill-qa/suite.json
Comment thread tests/e2e/specs/demo-homepage/demo-front-page.spec.ts Outdated
Comment thread tests/e2e/specs/woocommerce/header-cart.spec.ts Outdated
Demo front-page spec now fails on console errors as well as page errors. The header-cart spec adds a product that is not already in the admin's persistent cart, so its cleanup can no longer remove a pre-existing line.
@iamprazol

Copy link
Copy Markdown
Contributor Author

@tg-autopilot review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Unresolved test correctness, setup-gating, cleanup, and QA documentation issues remain.

Review effort: Lite
Findings: None

Resolved since last review (5)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Clean up cart items in a finally block

tests/​e2e/​specs/​woocommerce/​header-cart.spec.ts:64

Because this uses a saved admin account and its cart persists between runs, any failure in the count assertion or navigation before the cleanup leaves the newly added product in that account. Wrap the add/assert sequence in try/finally and perform the line removal in the finally block so a failed test cannot contaminate later runs.

A failed count assertion or navigation no longer leaves the added product in the saved admin account's persistent cart.
@iamprazol

Copy link
Copy Markdown
Contributor Author

Addressed Copilot's "Clean up cart items in a finally block" finding in 4498f2a. The add/assert sequence now sits in try/finally, and the new cart line is removed in the finally block. Verified that a forced assertion failure leaves the admin cart unchanged.

@iamprazol

Copy link
Copy Markdown
Contributor Author

@tg-autopilot review

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Two moderate issues remain in the Playwright dependency compatibility and cart cleanup logic.

Review effort: Lite
Findings: None

@iamprazol
iamprazol merged commit a29863f into master Sep 24, 2026
1 check passed
@iamprazol iamprazol self-assigned this Sep 24, 2026
@iamprazol
iamprazol deleted the add/claudegrill-qa-setup branch September 24, 2026 12:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants