web-qa

An AI agent skill that takes a URL, development code, or a brief feature description and guides the agent through real UI exploration, user-action scenarios, environment-specific E2E execution, and evidence-based reporting.

This is a procedure for coding agents with browser, file, and command execution tools, not a standalone test runner. Users do not need to write test cases or selectors in advance: the agent investigates UI elements and states, then connects scenarios, tests, and evidence.

Install with npx (recommended)

Node.js, npm (including npx), and Git are required. The skills CLI installs the skill directly from this GitHub repository; no separate web-qa npm package or Homebrew installation is needed.

Choose one installation scope below. You can select the target agent during installation.

Project — use in the current project

Run from the root of the project where you want to use the skill.

npx skills@latest add chonamdoo/web-qa-skill --skill web-qa

Global — use across projects

npx skills@latest add chonamdoo/web-qa-skill --skill web-qa --global

To target a particular agent, use its CLI agent ID. For example, to install for Codex in the current project:

npx skills@latest add chonamdoo/web-qa-skill --skill web-qa --agent codex

To list skills without installing:

npx skills@latest add chonamdoo/web-qa-skill --list

These GitHub commands become usable after the skill files are published to the repository's default branch. To inspect local sources, run npx skills@latest add . --list from this repository root.

Installing the skill does not automatically install Playwright, SafariDriver, Appium, Android SDK, or Xcode, or deploy an app. Prepare actual test tools and execution permissions separately.

Capabilities

Capability	Coverage
Three input paths	Explore real screens from a URL, investigate routes/state/execution configuration from code, or draft scenarios from a brief description.
Real UI investigation	Combine DOM, accessibility tree, and screen observations; inspect below-the-fold content, menus, tabs, modals, bottom sheets, errors, loading, and permission-dependent states.
User-action scenarios	Organize discovered clicks, taps, input, editing, selection, cancellation, submission, retries, and back navigation into normal, error, boundary, and sequential flows.
Appearance and interaction	Check text, images, icons, and controls for clipping, obstruction, overflow, keyboard access, focus, scrolling, and actual tap outcomes.
Browser/screen matrix	Test desktop Chrome/Safari, multiple CSS viewports, product breakpoint boundaries, and orientation changes.
Android/iOS behavior	Check system navigation areas, safe areas/home indicators, browser toolbars, software keyboards, rotation, and back behavior.
Mac localhost connections	Verify iOS Simulator host access, Android adb reverse, and separate API, authentication, and WebSocket connections.
Traceable results	Link UI elements → actions → scenarios → environment-specific results/evidence; expose unwritten, unexecuted, and missing combinations.

A small smoke subset does not replace discovered actions. The skill also does not promise exhaustive verification of infinitely many inputs, repetitions, and action sequences. Reports disclose investigated scope, boundary selection, inaccessible screens, and remaining work.

Usage examples

Ask the agent where the skill is installed using prompts such as these. Skill invocation and selection interfaces vary by host.

Chrome-only execution

Use web-qa to test https://example.test on desktop Chrome only.
Report the Chrome results; Safari and mobile are outside this run's scope.

Web app running on localhost

Use the web-qa skill to QA http://localhost:3000.
Explore real UI elements and states, write scenarios for user actions,
and execute them on desktop Chrome/Safari and Android/iOS.
Report bottom-area, keyboard, and rotation issues, plus unexecuted items.

Project with development code

Use the web-qa skill to find how to run this project and prepare the app.
Inspect the real UI, then write and execute scenarios and E2E tests.
Follow the project's existing test tools and execution rules.

Repeat analyzed scenarios through CLI

Use web-qa to rerun the analyzed scenarios on desktop Chrome only.
Validate the existing executable tests and isolated setup, then run them
in batches through the project's existing CLI runner.
Return a compact result index with links to full protected evidence.
Reinspect changed or ambiguous states and unresolved visual judgments.

CLI batching avoids a model decision at every action; it does not reduce required coverage or evidence. No OpenCLI dependency or new runner is bundled. Total token savings depend on test authoring, maintenance, image review, and failure analysis, not just output size.

Check CLI results and critical tests

Keep an existing compliant QA gate. Otherwise, the optional dependency-free checker validates required scenario/target records, all attempts, run/build identity, and evidence-file SHA-256 hashes:

node skills/web-qa/scripts/check-run.mjs PLAN.json RESULT.json EVIDENCE_ROOT

This path applies in this repository; in an application, resolve it from the installed skill directory. Node.js 18+ is required only for the checker. No Reticle or MCP installation is needed. Exit 0 means the supplied records and evidence integrity passed, not that the app or business outcomes were independently verified.

For critical persistence, authorization, or payment E2E tests, use isolated negative controls to check that a representative defect makes the unchanged test fail for the intended reason. Preserve the clean baseline, injected-fault failure, and restored baseline separately.

See CLI verification and false-green checks for the input contract, trust limits, safe fault injection, and cleanup.

Bounded exploration and attempt isolation

Model-directed exploration and recovery use recorded finite limits and scenario-specific waiting/progress conditions, not a universal repeated-click threshold. Limits do not shrink required coverage or turn unfinished work into PASS. Independent attempts verify their required client, account/server, and device starting state using existing fixtures; declared journeys may intentionally share state within their boundary. Scenario paths remain part of the test: an alternative route cannot erase a failed required step.

Final QA reconciliation includes required teardown/restoration and run-level errors, even if assertions and the records/evidence checker passed. Original failures and cleanup errors remain distinct; optional diagnostic failures alone do not invalidate otherwise complete QA. These are procedure rules, not a bundled isolation engine or automatic enforcement by check-run.mjs.

Feature described without an app

Use the web-qa skill to draft QA scenarios for an event registration screen.
Capacity is four people, and duplicate registration by one person is prohibited.
Confirming cancellation must free one place. There is no runnable app yet.

With only a description and no runnable app, produce scenario drafts only and leave real UI observation and E2E unexecuted. Distinguish observed current behavior from expected results grounded in requirements.

Environment requirements for actual testing

Prefer the project's existing test tools. Defaults for a new setup are:

Target	Default path
Desktop Chrome	Playwright Test with the actual Chrome channel
Desktop Safari	macOS Safari + safaridriver/WebDriver
Android Chrome	Android Emulator or device + Appium UiAutomator2
iOS Safari	iOS Simulator or device + Appium XCUITest
Internet Explorer	Test in an available legacy environment; otherwise exclude with a reason.

Playwright WebKit does not replace actual Safari testing, and mobile viewport settings do not replace Android/iOS OS testing. Distinguish Simulator/Emulator results from physical-device results. Verify specific browser, OS, device, and driver versions in the target project and installed environment.

• Prepare test accounts, synthetic data, and an authorized workspace. Follow the project's worktree and build/test rules when executing code.
• Payments, sending, publishing, real data changes, and external transfers require an explicitly authorized scope.
• Cloud devices, public tunnels, and additional tool installation are not automatically authorized.
• Judge PASS against the explicitly requested scope, or the default matrix when none is specified. Chrome-only runs do not require Safari or mobile execution. Missing required targets still prevent PASS; tool unavailability alone does not authorize excluding them.

See Browser and mobile testing and Localhost connections for preparation details.

Deliverables and judgments

Use the project's existing formats for UI element/action inventories, scenarios, executable tests, environment-specific results, reproduction steps, and screen/log evidence.

Distinguish PASS, FAIL, FLAKY, SKIPPED, BLOCKED, and NOT_RUN. Generated test code, successful click commands, or a successful final retry alone do not establish an overall pass. Separate product defects, environment issues, missing information, and approved exclusions.

Repository structure

skills/web-qa/
├── SKILL.md
├── scripts/
│   └── check-run.mjs
└── references/
    ├── cli-verification.md
    ├── mobile-and-browsers.md
    ├── localhost.md
    └── sources.md

• SKILL.md: shared execution procedure and completion criteria.
• sources.md: user requirements, author-derived defaults, official technical sources, and applicability limits.

Validation scope

The original PR #1 material at commit 38f1465 reports checks of frontmatter, internal references, and package boundaries, plus synthetic evaluations by a separate executor for description-based scenario drafting and failure/gap judgments from execution records.

That material also reports running the actual skills CLI add … --list against the local repository and finding one skill, web-qa. Listing is not installation or verification of publication to the remote default branch.

Those historical reports are preserved as provenance, not claimed as fresh execution of this English revision. They do not establish real browser interaction, execution of generated tests, Appium/device connectivity, or automatic discovery on every host. Verify the actual tools and environments when using the skill on a product.

The CLI checker has dependency-free regression tests runnable with node --test tests/check-run.test.mjs. They exercise result completeness, retries, run/build mismatches, evidence integrity, and path containment using isolated synthetic files. They do not validate a product's browser flows or certify the truth of reporter-supplied results.

Evaluate each skill update

The reusable evaluation package in evals/README.md contains two synthetic shopping pages, ten criteria, six planning/context cases, and fresh URL-only inputs. The pages cover conditional controls, scrolling, input validation, and layout checks. Evaluator answers must remain separate from the fresh executor.

Where	What runs	What it proves
GitHub CI	npm run eval:static: build, typecheck, scoped asset lint, and gate-contract regression tests	Assets and report-validation rules pass. It does not run the skill or certify browser behavior.
Local desktop	Fresh skill execution on actual Chrome and Safari at 1024/1280, followed by npm run eval:check with current report/evidence	Supplied current-version records meet the behavioral gate. An independent evaluator must still verify that the recorded actions and outcomes occurred.

After every skill or evaluation update:

1. Run npm ci, then npm run eval:static.
2. Start npm run eval:serve and freeze npm run eval:fingerprint before execution.
3. Follow the local evaluation procedure: give a fresh executor only the current skill and opaque page URL, generate user-focused scenarios from the rendered UI, and execute the same suite on both fixtures. Evaluate planning/context cases separately.
4. Preserve all attempts and nonsecret evidence. Run npm run eval:check with the current report and evidence root; review its result before treating the update as behaviorally verified.

Missing evidence stays BLOCKED; stale, incomplete or failing evidence cannot PASS. Expected seeded faults do not excuse unrelated failing assertions: every additional reported assertion must PASS on both fixtures. GitHub CI does not execute or validate a local behavioral report, and CI success alone is not proof of behavioral improvement.

Mobile is excluded from this repository evaluation suite. Playwright WebKit does not certify actual Safari. Automatic skill learning and rewriting are not implemented; proposed lessons require independent behavioral replay before changing the shared skill. Byte measurements do not establish token savings without measured token data.