Skip to content

test(frontend): add end-to-end test suite - #195

Merged
andre8244 merged 29 commits into
mainfrom
e2e-testing
Sep 25, 2026
Merged

andre8244 merged 29 commits into
mainfrom
e2e-testing

Conversation

@andre8244

@andre8244 andre8244 commented Sep 8, 2026 •

Copy link
Copy Markdown
Collaborator

TL;DR

  • What it adds: browser end-to-end tests (Playwright). It runs the real app against a real backend and signs in through the real Logto login form.
  • Where each part runs:
    • Full browser suite: on a PR only when the run-e2e label is added, plus every merge to main, on demand and weekly. It uses a Logto tenant reserved for CI.
    • QA smoke: read-only checks against QA after each deploy.
    • Production is never touched.
  • Merge rule: A PR can be merged only once the suite has passed on its last commit. PRs that change only documentation pass without a run.
  • Coverage:
    • Sign-in and the session.
    • All 21 organization role × technical role pairs (menus, sections they can open, buttons they're offered), plus hierarchy scoping of the list views.
    • Creating and editing companies, systems and users.

📋 Description

Adds browser-level end-to-end tests. The API is already covered
end-to-end by the authorization suite, but nothing exercised the UI.

The suite drives the real application against a real backend, signing in
through the actual login form as the users the existing test fixture
already provides.

Where each layer runs

Layer When Against
Authorization spec check every push to main and every pull request targeting main nothing external — reads the repo's own files, about a second
Browser suite (fullstack) on a pull request when the run-e2e label is added, merge to main, on demand, weekly the whole stack built and booted on the runner, with a Logto tenant dedicated to CI
QA smoke (smoke) after each QA deploy the deployed QA environment, read-only

Production is never contacted. The CI tenant is separate from the
development and QA ones, so provisioning and removing its test users
collides with nobody.

Running it on a pull request

Add the run-e2e label to run the browser suite. The result is the
e2e/fullstack status on the latest commit; every push resets it, so a
pull request merges only once the suite passed on its last commit.
Documentation-only pull requests pass without a run.

Show tests included

Sign-in (2)

  • a saved session lands on the dashboard without a round trip to Logto
  • an unauthenticated visitor is sent to Logto

Permissions (42 — every organization role × technical role pair)

  • each pair is offered exactly the navigation its permissions allow
  • each pair reaches the sections it may read and is refused the rest

Action controls (21 — one per pair)

  • each pair is offered exactly the actions its permissions allow inside a
    page, which is where a hidden section still leaves a visible button

Hierarchy scoping (6)

  • /resellers and /customers list the caller's own branch and nothing
    outside it, from a distributor down to a reseller
  • /users lists a customer organization's own members only

Session (4)

  • a live session survives a reload without going back to Logto
  • a tab with no JWT pair re-mints one silently
  • a refusal offers a way back from the forbidden page
  • a refusal leaves the rest of the session usable

Organizations (3)

  • creates a company and lists it
  • refuses a company with no VAT number
  • edits a company and keeps the change

Systems (3)

  • lists a new system and filters down to it
  • opens the detail view for a system
  • stops offering to regenerate the secret once the system registers

Users (4)

  • creates a user with a role and lists it
  • changes the assigned role and keeps the change
  • archives the user
  • refuses a user with no role assigned

QA smoke (4, read-only, against the deployed environment)

  • serves a healthy backend
  • sends an anonymous visitor to the identity provider
  • signs in, loads the dashboard and signs out
  • renders a list page

Test data

The tests assume no company, user or system exists beforehand. Each one
creates what it needs, named under a reserved prefix, and removes it
afterwards; the cleanup refuses to touch anything outside that prefix,
and each file sweeps leftovers before it starts, so a run that crashed
earlier cannot influence the next. The database is new on every run.

What must be prepared once, per tenant, is the identity provider: an
initialized Logto tenant with its roles and its owner account. The
secrets are listed in .github/workflows/README.md.

Notes

Three changes reach the application code; two of them change what
production builds do:

  • Pinia Colada devtools out of production builds. App.vue mounted
    PiniaColadaProdDevtools, which ships the panel in every build. It now
    mounts PiniaColadaDevtools, which is dev-only, so the panel no longer
    appears on qa or production.
  • Re-authentication keeps the current page. When the session dies
    mid-visit, stores/login.ts saves the page the user is on before
    redirecting to Logto, so they come back there instead of on the
    dashboard.
  • A build-time VITE_E2E flag, set only for test builds, disables
    background refetching and the devtools panel, because both interfere
    with the tests. It has no effect on dev, qa or production builds.

Also adds a cheap authorization check to the existing pipeline: it fails
when an endpoint ships without a declared permission, or when the test
fixture names a role no configuration defines.

@andre8244 andre8244 self-assigned this Sep 8, 2026
@edospadoni
edospadoni deployed to e2e-testing - my-frontend-qa PR #195 September 8, 2026 14:05 — with Render Active
@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

🔗 Redirect URIs Added to Logto

The following redirect URIs have been automatically added to the Logto application configuration:

Redirect URIs:

  • https://my-proxy-qa-pr-195.onrender.com/login-redirect

Post-logout redirect URIs:

  • https://my-proxy-qa-pr-195.onrender.com/login

These will be automatically removed when the PR is closed or merged.

@edospadoni
edospadoni deployed to e2e-testing - my-frontend-qa PR #195 September 8, 2026 15:28 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-frontend-qa PR #195 September 9, 2026 08:03 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-backend-qa PR #195 September 9, 2026 08:04 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-frontend-qa PR #195 September 9, 2026 08:20 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-frontend-qa PR #195 September 9, 2026 08:45 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-backend-qa PR #195 September 9, 2026 08:45 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-backend-qa PR #195 September 9, 2026 09:53 — with Render Active
@andre8244 andre8244 changed the title test(frontend): add Playwright end-to-end suite test(frontend): add end-to-end test suite Sep 9, 2026
@edospadoni
edospadoni deployed to e2e-testing - my-frontend-qa PR #195 September 9, 2026 18:12 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-frontend-qa PR #195 September 10, 2026 09:43 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-frontend-qa PR #195 September 10, 2026 09:53 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-frontend-qa PR #195 September 10, 2026 10:03 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-backend-qa PR #195 September 10, 2026 10:03 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-frontend-qa PR #195 September 10, 2026 10:23 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-frontend-qa PR #195 September 11, 2026 06:00 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-backend-qa PR #195 September 11, 2026 06:00 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-frontend-qa PR #195 September 14, 2026 06:00 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-backend-qa PR #195 September 14, 2026 06:00 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-frontend-qa PR #195 September 15, 2026 06:00 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-backend-qa PR #195 September 15, 2026 06:00 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-frontend-qa PR #195 September 16, 2026 06:00 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-backend-qa PR #195 September 16, 2026 06:01 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-frontend-qa PR #195 September 17, 2026 06:00 — with Render Active
The other half of "the UI hides the button" that navigation coverage
cannot reach. rbac.spec.ts proves a persona is offered the right
sections; this proves that once inside one it is offered the right
actions — a Reader may open /systems and must not be invited to create
one, a distributor Admin manages resellers and must not be offered
Promote, which moves an organization out of its own reach.

Expectations are hand-authored, as in the RBAC spec: SURFACES states
what each control should require, from the vocabulary in
lib/permissions.ts and the documented intent of each gate. Copying the
table components' conditions in would prove nothing.

Every surface names a control that must be present for anyone who can
open it, so a page that failed to render — or a kebab that failed to
open — cannot satisfy the absences beneath it. Reachability is a
conjunction rather than a single read permission, because the reseller
kebab also needs whatever puts an item in the menu: the backend
subtracts the organization role's manage:* from anyone holding Reader
(authz/model.yml, filterManagePermissionsForReader), so a distributor
Reader has no kebab at all and nothing to assert about. What it is
denied instead is asserted on the create button on the same page.

Impersonation and the add-on catalog are left out and tracked: both need
a fixture this spec would have to build, and the second includes
canBuyAddons — manage:entitlements AND NOT owner-level, the only gate
here where holding more authority has to hide a control.

Assisted-by: Claude Code:Opus 5 (1M context)
stores/login.ts and the interceptors in lib/axios.ts are the most
breakage-prone code in the application — a rotating refresh chain, a
sessionStorage-backed JWT pair, a 401 that replays once and a 403 that
redirects — and no spec exercised any of it. Every other spec depends on
it and none asserted it.

Four questions: a reload re-exchanges in place rather than re-entering
the sign-in flow; a tab with no JWT pair re-mints one silently, which is
the state every spec actually starts in and was relied on rather than
asserted; /forbidden offers a way back and it works; and a 403 leaves
the rest of the session usable, since an interceptor that turned one
missing permission into a logout is precisely the kind of thing an
API-level suite cannot see.

No sign-out test, deliberately. Signing out ends the persona's Logto SSO
session rather than just this tab's, so the saved storageState every
other spec boots from would be dead and which spec noticed first would
depend on worker scheduling. It needs a persona of its own, or a project
that runs last.

Assisted-by: Claude Code:Opus 5 (1M context)
Adding the pull_request trigger left three places describing the
previous behaviour: e2e-main.yml's own header, fifteen lines above the
block it contradicts, and both READMEs.

The concurrency comment also had the cost wrong. GitHub keeps at most
one run pending per group and cancels the previously pending one, so the
queue never grows and a burst does not serialize. What a burst does is
skip the commits in the middle, which is a partial retreat from the
per-commit attribution the trigger exists for — worth documenting,
unlike a queue that cannot happen. Cancelling instead would trade it for
abandoned organizations.

paths-ignore was on pull_request only, so a docs-only merge to main
still built four images and took the global lock.

Four secrets were interpolated straight into a run: body while the rest
of the step passes them through env:; they go through env: now too.

The tenant was described two ways — "the development tenant" in the
workflow README against a tenant "dedicated to CI" in the pull request.
Settled in favour of the latter, with the reason it matters: prefix in
authz/fixture.yml fixes the organization keys and persona addresses, so
CI and a local `authz provision` on one tenant fight over the same Logto
users. Note that what each side *deletes* is safely scoped either way —
the specs refuse any name outside the e2e- prefix and authz teardown
only removes what its own registry records — so the failure mode is a
collision while provisioning, not lost data.

Records that the job carries no SMTP_* on purpose: creating a user sends
a welcome email with a temporary password, the users spec creates one per
run, and with no relay configured SendEmail fails closed before touching
the network.

Also documents that a bare `npm run test:e2e` cannot pass locally, since
it includes the smoke project, and adds the two lessons from this round
to the gotchas: a negative assertion needs a positive control, and
waiting for something already on screen proves nothing.

Assisted-by: Claude Code:Opus 5 (1M context)
IS_E2E was imported in a second statement from './lib/config', three
lines below the one that already imports from it.

Assisted-by: Claude Code:Opus 5 (1M context)
A single config served three projects with incompatible needs, and the
variables steering it had grown two meanings each.

E2E_BASE_URL named the origin and, by being set at all, suppressed the
webServer block. e2e-main.yml set it to http://localhost:5173 — the same
string the config already defaulted to — so only the suppression was
ever wanted there. E2E_SMOKE_BASE_URL then had to exist because
E2E_BASE_URL would have dragged the mutating fullstack project onto a
deployed environment along with smoke, leaving three sources for one
value: that variable, E2E_BASE_URL, and the QA default in the config.
Skipping the dev server for a smoke-only run meant reading --project out
of process.argv, which is a workaround for one file serving two targets
rather than a thing worth doing.

Split instead. playwright.config.ts keeps setup and fullstack, pinned to
http://localhost:5173 with no env read at all — the Logto fixture only
accepts that origin as a redirect URI, so it was never configurable in
the first place. playwright.config.smoke.ts holds the deployed suite,
with no webServer and no setup dependency, and is the only place a base
URL can still be overridden. Options common to both moved to
e2e/playwright.shared.ts, so the split duplicates nothing.

What replaces E2E_BASE_URL in CI is reuseExistingServer: true. Compose
already publishes the proxy on 5173, so Playwright finds it listening
and starts nothing; locally it starts dev:e2e itself. The known cost is
that a plain `npm run dev` is reused just as readily and does not carry
VITE_E2E — already true before this change, and now stated where the
option is set.

Both scripts work with no variable set: `npm run test:e2e` for the local
suite, `npm run test:e2e:smoke` for QA. The name is
playwright.config.smoke.ts rather than playwright.smoke.config.ts so the
existing tsconfig.e2e.json include, playwright.config.*, keeps matching.

Assisted-by: Claude Code:Opus 5 (1M context)
App.vue rendered PiniaColadaProdDevtools, which is the variant that
deliberately ships the devtools panel in production builds — the
opposite of what was wanted. Every qa and production bundle carried the
whole panel: 97 occurrences of its splitpanes UI and roughly 300 KB of
uncompressed assets, on a page no user can open it from.

PiniaColadaDevtools is the dev-only export. It resolves to an empty
wrapper unless NODE_ENV is development, so the panel is gone from the
built bundle rather than merely hidden.

The v-if stays. `npm run dev:e2e` is a development build, where the
panel does mount and injects itself into the DOM, shadowing e2e
selectors — which is the case VITE_E2E exists for.

Assisted-by: Claude Code:Opus 5 (1M context)
The full-stack job never set LOGTO_API_RESOURCE, so the backend refused to
start and the stack never came up. The same value is needed in three places,
because all three have to agree on the audience of the access token: the
backend that validates it, the frontend build that requests it, and the
apitool registry, whose login would otherwise obtain an opaque token that
/auth/exchange refuses.

docker-compose.yml never forwarded VITE_LOGTO_API_RESOURCE to the frontend
build either, although the Containerfile declares the argument, so any
compose-built frontend asked for a token with no resource at all.

Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Second variable the backend refuses to start without: the token exchange
binds the JWT to that client, so it is fatal like the resource indicator.
It is the application id the fixture is already provisioned against, so it
comes from E2E_LOGTO_APP_ID rather than a new secret.

The rest of the fatal configuration is now covered: DATABASE_URL and
REDIS_URL come from compose, everything else from this file.

Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
… refuses

The fixture login failed as "session not found": Logto had refused the
authorization request naming an API resource the CI tenant does not have, and
a refused request sets no interaction cookie, so every later call reported the
missing cookie instead of the reason. Nothing creates that resource — `sync`
is never run against this tenant — so the job now runs it over config.ci.yml
before provisioning, with API_BASE_URL derived from the indicator so the
resource it creates is the audience the backend, the SPA and apitool share.

apitool no longer discards the status of the authorization request or of the
interaction start, so a refusal is reported where it happens.

Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Provisioning refused the distributor: the backend rejects a portal list naming
an application the tenant does not have, and the CI vocabulary described none,
so the tenant only ever had what someone had registered by hand.

The six applications now come from the same place as the rest of the
vocabulary, taken from the QA config. NethStock keeps its organization_ids
pin with an id that matches no organization here: apps.yml expects it to be
visible to nobody wherever the config was not written for, and real ids stay
out of a tracked file.

Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
A re-auth started by the store is a full-page redirect, and LoginRedirectView
only resumes what the router guard saved — which is nothing, because the guard
never ran. So a session that dies mid-visit reappears on the dashboard, and
the user loses the page they were reading.

Save the same key the guard saves, with the same exclusions. It also stops the
e2e suite reporting this as a product bug: with the audience binding every page
boot exchanges the refresh token for a resource-scoped one, Logto rotates it,
so the token saved in e2e/.auth is good for a single boot and every later spec
takes exactly this path.

Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
A registry written before /auth/exchange required aud=LOGTO_API_RESOURCE has
no logto_resource, so apitool logs in without one, Logto issues an opaque
token and the backend answers "invalid access token". Nothing pointed at the
registry, so the fault read as the backend's.

Say it once per run, at the only place every command goes through.

Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Refresh token rotation off on the SPA is now a requirement of every tenant the
suite drives, and nothing in the repository can express it. Left unwritten,
the next tenant hits the same wall this branch did: invalid_grant on the token
endpoint, surfacing as specs timing out on /auth/exchange, which reads as a
slow application rather than as tenant configuration.

Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
A pull request runs the full-stack suite only when the run-e2e label
is added, and the run reports its verdict as the e2e/fullstack commit
status on the head commit. e2e-gate.yml resets that status to pending
on every push (success for documentation-only pull requests), so with
the status required on main a pull request merges only after the suite
passed on its last commit. Push to main, dispatch and the weekly cron
keep running it as they do.

Assisted-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@edospadoni
edospadoni deployed to e2e-testing - my-backend-qa PR #195 September 23, 2026 15:57 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-frontend-qa PR #195 September 23, 2026 15:57 — with Render Active
@andre8244 andre8244 added the run-e2e Run the full-stack browser suite on this PR label Sep 23, 2026
@github-actions github-actions Bot removed the run-e2e Run the full-stack browser suite on this PR label Sep 23, 2026
@edospadoni
edospadoni deployed to e2e-testing - my-frontend-qa PR #195 September 24, 2026 06:01 — with Render Active
@edospadoni
edospadoni deployed to e2e-testing - my-backend-qa PR #195 September 24, 2026 06:01 — with Render Active
@andre8244 andre8244 added the run-e2e Run the full-stack browser suite on this PR label Sep 24, 2026
@github-actions github-actions Bot removed the run-e2e Run the full-stack browser suite on this PR label Sep 24, 2026
@edospadoni
edospadoni temporarily deployed to e2e-testing - my-frontend-qa PR #195 September 25, 2026 06:01 — with Render Destroyed
@edospadoni
edospadoni temporarily deployed to e2e-testing - my-backend-qa PR #195 September 25, 2026 06:01 — with Render Destroyed
@andre8244
andre8244 marked this pull request as ready for review September 25, 2026 09:35
@andre8244
andre8244 merged commit ba1837e into main Sep 25, 2026
16 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

🗑️ Redirect URIs Removed from Logto

The following redirect URIs have been automatically removed from the Logto application configuration:

Redirect URIs:

  • https://my-proxy-qa-pr-195.onrender.com/login-redirect

Post-logout redirect URIs:

  • https://my-proxy-qa-pr-195.onrender.com/login

Cleanup completed for PR #195.

edospadoni added a commit that referenced this pull request Sep 25, 2026
PR #195 split the frontend "test" script into test:unit and
test:e2e; release.sh still called the old name and failed with
"Missing script".

This branch was successfully deployed

No deployments
e2e-testing - my-backend-qa PR #195 — 2cf2c22d Deployed Sep 25, 2026 by edospadoni
e2e-testing - my-frontend-qa PR #195 — 2cf2c22d Deployed Sep 25, 2026 by edospadoni
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants