test(frontend): add end-to-end test suite - #195
Merged
Merged
Conversation
edospadoni
deployed
to
e2e-testing - my-frontend-qa PR #195
September 8, 2026 14:05 — with
Render
Active
Contributor
|
🔗 Redirect URIs Added to Logto The following redirect URIs have been automatically added to the Logto application configuration: Redirect URIs:
Post-logout redirect URIs:
These will be automatically removed when the PR is closed or merged. |
andre8244
force-pushed
the
e2e-testing
branch
from
September 8, 2026 15:28
d099a26 to
aaa2586
Compare
edospadoni
deployed
to
e2e-testing - my-frontend-qa PR #195
September 8, 2026 15:28 — with
Render
Active
andre8244
force-pushed
the
e2e-testing
branch
from
September 9, 2026 08:03
935af0c to
2806126
Compare
edospadoni
deployed
to
e2e-testing - my-frontend-qa PR #195
September 9, 2026 08:03 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-backend-qa PR #195
September 9, 2026 08:04 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-frontend-qa PR #195
September 9, 2026 08:20 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-frontend-qa PR #195
September 9, 2026 08:45 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-backend-qa PR #195
September 9, 2026 08:45 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-backend-qa PR #195
September 9, 2026 09:53 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-frontend-qa PR #195
September 9, 2026 18:12 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-frontend-qa PR #195
September 10, 2026 09:43 — with
Render
Active
andre8244
force-pushed
the
e2e-testing
branch
from
September 10, 2026 09:53
6233046 to
90b8127
Compare
edospadoni
deployed
to
e2e-testing - my-frontend-qa PR #195
September 10, 2026 09:53 — with
Render
Active
andre8244
force-pushed
the
e2e-testing
branch
from
September 10, 2026 10:03
90b8127 to
0da2da8
Compare
edospadoni
deployed
to
e2e-testing - my-frontend-qa PR #195
September 10, 2026 10:03 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-backend-qa PR #195
September 10, 2026 10:03 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-frontend-qa PR #195
September 10, 2026 10:23 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-frontend-qa PR #195
September 11, 2026 06:00 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-backend-qa PR #195
September 11, 2026 06:00 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-frontend-qa PR #195
September 14, 2026 06:00 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-backend-qa PR #195
September 14, 2026 06:00 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-frontend-qa PR #195
September 15, 2026 06:00 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-backend-qa PR #195
September 15, 2026 06:00 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-frontend-qa PR #195
September 16, 2026 06:00 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-backend-qa PR #195
September 16, 2026 06:01 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-frontend-qa PR #195
September 17, 2026 06:00 — with
Render
Active
The other half of "the UI hides the button" that navigation coverage cannot reach. rbac.spec.ts proves a persona is offered the right sections; this proves that once inside one it is offered the right actions — a Reader may open /systems and must not be invited to create one, a distributor Admin manages resellers and must not be offered Promote, which moves an organization out of its own reach. Expectations are hand-authored, as in the RBAC spec: SURFACES states what each control should require, from the vocabulary in lib/permissions.ts and the documented intent of each gate. Copying the table components' conditions in would prove nothing. Every surface names a control that must be present for anyone who can open it, so a page that failed to render — or a kebab that failed to open — cannot satisfy the absences beneath it. Reachability is a conjunction rather than a single read permission, because the reseller kebab also needs whatever puts an item in the menu: the backend subtracts the organization role's manage:* from anyone holding Reader (authz/model.yml, filterManagePermissionsForReader), so a distributor Reader has no kebab at all and nothing to assert about. What it is denied instead is asserted on the create button on the same page. Impersonation and the add-on catalog are left out and tracked: both need a fixture this spec would have to build, and the second includes canBuyAddons — manage:entitlements AND NOT owner-level, the only gate here where holding more authority has to hide a control. Assisted-by: Claude Code:Opus 5 (1M context)
stores/login.ts and the interceptors in lib/axios.ts are the most breakage-prone code in the application — a rotating refresh chain, a sessionStorage-backed JWT pair, a 401 that replays once and a 403 that redirects — and no spec exercised any of it. Every other spec depends on it and none asserted it. Four questions: a reload re-exchanges in place rather than re-entering the sign-in flow; a tab with no JWT pair re-mints one silently, which is the state every spec actually starts in and was relied on rather than asserted; /forbidden offers a way back and it works; and a 403 leaves the rest of the session usable, since an interceptor that turned one missing permission into a logout is precisely the kind of thing an API-level suite cannot see. No sign-out test, deliberately. Signing out ends the persona's Logto SSO session rather than just this tab's, so the saved storageState every other spec boots from would be dead and which spec noticed first would depend on worker scheduling. It needs a persona of its own, or a project that runs last. Assisted-by: Claude Code:Opus 5 (1M context)
Adding the pull_request trigger left three places describing the previous behaviour: e2e-main.yml's own header, fifteen lines above the block it contradicts, and both READMEs. The concurrency comment also had the cost wrong. GitHub keeps at most one run pending per group and cancels the previously pending one, so the queue never grows and a burst does not serialize. What a burst does is skip the commits in the middle, which is a partial retreat from the per-commit attribution the trigger exists for — worth documenting, unlike a queue that cannot happen. Cancelling instead would trade it for abandoned organizations. paths-ignore was on pull_request only, so a docs-only merge to main still built four images and took the global lock. Four secrets were interpolated straight into a run: body while the rest of the step passes them through env:; they go through env: now too. The tenant was described two ways — "the development tenant" in the workflow README against a tenant "dedicated to CI" in the pull request. Settled in favour of the latter, with the reason it matters: prefix in authz/fixture.yml fixes the organization keys and persona addresses, so CI and a local `authz provision` on one tenant fight over the same Logto users. Note that what each side *deletes* is safely scoped either way — the specs refuse any name outside the e2e- prefix and authz teardown only removes what its own registry records — so the failure mode is a collision while provisioning, not lost data. Records that the job carries no SMTP_* on purpose: creating a user sends a welcome email with a temporary password, the users spec creates one per run, and with no relay configured SendEmail fails closed before touching the network. Also documents that a bare `npm run test:e2e` cannot pass locally, since it includes the smoke project, and adds the two lessons from this round to the gotchas: a negative assertion needs a positive control, and waiting for something already on screen proves nothing. Assisted-by: Claude Code:Opus 5 (1M context)
IS_E2E was imported in a second statement from './lib/config', three lines below the one that already imports from it. Assisted-by: Claude Code:Opus 5 (1M context)
A single config served three projects with incompatible needs, and the variables steering it had grown two meanings each. E2E_BASE_URL named the origin and, by being set at all, suppressed the webServer block. e2e-main.yml set it to http://localhost:5173 — the same string the config already defaulted to — so only the suppression was ever wanted there. E2E_SMOKE_BASE_URL then had to exist because E2E_BASE_URL would have dragged the mutating fullstack project onto a deployed environment along with smoke, leaving three sources for one value: that variable, E2E_BASE_URL, and the QA default in the config. Skipping the dev server for a smoke-only run meant reading --project out of process.argv, which is a workaround for one file serving two targets rather than a thing worth doing. Split instead. playwright.config.ts keeps setup and fullstack, pinned to http://localhost:5173 with no env read at all — the Logto fixture only accepts that origin as a redirect URI, so it was never configurable in the first place. playwright.config.smoke.ts holds the deployed suite, with no webServer and no setup dependency, and is the only place a base URL can still be overridden. Options common to both moved to e2e/playwright.shared.ts, so the split duplicates nothing. What replaces E2E_BASE_URL in CI is reuseExistingServer: true. Compose already publishes the proxy on 5173, so Playwright finds it listening and starts nothing; locally it starts dev:e2e itself. The known cost is that a plain `npm run dev` is reused just as readily and does not carry VITE_E2E — already true before this change, and now stated where the option is set. Both scripts work with no variable set: `npm run test:e2e` for the local suite, `npm run test:e2e:smoke` for QA. The name is playwright.config.smoke.ts rather than playwright.smoke.config.ts so the existing tsconfig.e2e.json include, playwright.config.*, keeps matching. Assisted-by: Claude Code:Opus 5 (1M context)
App.vue rendered PiniaColadaProdDevtools, which is the variant that deliberately ships the devtools panel in production builds — the opposite of what was wanted. Every qa and production bundle carried the whole panel: 97 occurrences of its splitpanes UI and roughly 300 KB of uncompressed assets, on a page no user can open it from. PiniaColadaDevtools is the dev-only export. It resolves to an empty wrapper unless NODE_ENV is development, so the panel is gone from the built bundle rather than merely hidden. The v-if stays. `npm run dev:e2e` is a development build, where the panel does mount and injects itself into the DOM, shadowing e2e selectors — which is the case VITE_E2E exists for. Assisted-by: Claude Code:Opus 5 (1M context)
The full-stack job never set LOGTO_API_RESOURCE, so the backend refused to start and the stack never came up. The same value is needed in three places, because all three have to agree on the audience of the access token: the backend that validates it, the frontend build that requests it, and the apitool registry, whose login would otherwise obtain an opaque token that /auth/exchange refuses. docker-compose.yml never forwarded VITE_LOGTO_API_RESOURCE to the frontend build either, although the Containerfile declares the argument, so any compose-built frontend asked for a token with no resource at all. Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Second variable the backend refuses to start without: the token exchange binds the JWT to that client, so it is fatal like the resource indicator. It is the application id the fixture is already provisioned against, so it comes from E2E_LOGTO_APP_ID rather than a new secret. The rest of the fatal configuration is now covered: DATABASE_URL and REDIS_URL come from compose, everything else from this file. Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
… refuses The fixture login failed as "session not found": Logto had refused the authorization request naming an API resource the CI tenant does not have, and a refused request sets no interaction cookie, so every later call reported the missing cookie instead of the reason. Nothing creates that resource — `sync` is never run against this tenant — so the job now runs it over config.ci.yml before provisioning, with API_BASE_URL derived from the indicator so the resource it creates is the audience the backend, the SPA and apitool share. apitool no longer discards the status of the authorization request or of the interaction start, so a refusal is reported where it happens. Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Provisioning refused the distributor: the backend rejects a portal list naming an application the tenant does not have, and the CI vocabulary described none, so the tenant only ever had what someone had registered by hand. The six applications now come from the same place as the rest of the vocabulary, taken from the QA config. NethStock keeps its organization_ids pin with an id that matches no organization here: apps.yml expects it to be visible to nobody wherever the config was not written for, and real ids stay out of a tracked file. Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
A re-auth started by the store is a full-page redirect, and LoginRedirectView only resumes what the router guard saved — which is nothing, because the guard never ran. So a session that dies mid-visit reappears on the dashboard, and the user loses the page they were reading. Save the same key the guard saves, with the same exclusions. It also stops the e2e suite reporting this as a product bug: with the audience binding every page boot exchanges the refresh token for a resource-scoped one, Logto rotates it, so the token saved in e2e/.auth is good for a single boot and every later spec takes exactly this path. Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
A registry written before /auth/exchange required aud=LOGTO_API_RESOURCE has no logto_resource, so apitool logs in without one, Logto issues an opaque token and the backend answers "invalid access token". Nothing pointed at the registry, so the fault read as the backend's. Say it once per run, at the only place every command goes through. Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Refresh token rotation off on the SPA is now a requirement of every tenant the suite drives, and nothing in the repository can express it. Left unwritten, the next tenant hits the same wall this branch did: invalid_grant on the token endpoint, surfacing as specs timing out on /auth/exchange, which reads as a slow application rather than as tenant configuration. Assisted-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
A pull request runs the full-stack suite only when the run-e2e label is added, and the run reports its verdict as the e2e/fullstack commit status on the head commit. e2e-gate.yml resets that status to pending on every push (success for documentation-only pull requests), so with the status required on main a pull request merges only after the suite passed on its last commit. Push to main, dispatch and the weekly cron keep running it as they do. Assisted-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
andre8244
force-pushed
the
e2e-testing
branch
from
September 23, 2026 15:57
671c504 to
2cf2c22
Compare
edospadoni
deployed
to
e2e-testing - my-backend-qa PR #195
September 23, 2026 15:57 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-frontend-qa PR #195
September 23, 2026 15:57 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-frontend-qa PR #195
September 24, 2026 06:01 — with
Render
Active
edospadoni
deployed
to
e2e-testing - my-backend-qa PR #195
September 24, 2026 06:01 — with
Render
Active
edospadoni
temporarily deployed
to
e2e-testing - my-frontend-qa PR #195
September 25, 2026 06:01 — with
Render
Destroyed
edospadoni
temporarily deployed
to
e2e-testing - my-backend-qa PR #195
September 25, 2026 06:01 — with
Render
Destroyed
andre8244
marked this pull request as ready for review
September 25, 2026 09:35
Contributor
|
🗑️ Redirect URIs Removed from Logto The following redirect URIs have been automatically removed from the Logto application configuration: Redirect URIs:
Post-logout redirect URIs:
Cleanup completed for PR #195. |
edospadoni
added a commit
that referenced
this pull request
Sep 25, 2026
PR #195 split the frontend "test" script into test:unit and test:e2e; release.sh still called the old name and failed with "Missing script".
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR
📋 Description
Adds browser-level end-to-end tests. The API is already covered
end-to-end by the authorization suite, but nothing exercised the UI.
The suite drives the real application against a real backend, signing in
through the actual login form as the users the existing test fixture
already provides.
Where each layer runs
mainand every pull request targetingmainfullstack)run-e2elabel is added, merge tomain, on demand, weeklysmoke)Production is never contacted. The CI tenant is separate from the
development and QA ones, so provisioning and removing its test users
collides with nobody.
Running it on a pull request
Add the
run-e2elabel to run the browser suite. The result is thee2e/fullstackstatus on the latest commit; every push resets it, so apull request merges only once the suite passed on its last commit.
Documentation-only pull requests pass without a run.
Show tests included
Sign-in (2)
Permissions (42 — every organization role × technical role pair)
Action controls (21 — one per pair)
page, which is where a hidden section still leaves a visible button
Hierarchy scoping (6)
outside it, from a distributor down to a reseller
Session (4)
Organizations (3)
Systems (3)
Users (4)
QA smoke (4, read-only, against the deployed environment)
Test data
The tests assume no company, user or system exists beforehand. Each one
creates what it needs, named under a reserved prefix, and removes it
afterwards; the cleanup refuses to touch anything outside that prefix,
and each file sweeps leftovers before it starts, so a run that crashed
earlier cannot influence the next. The database is new on every run.
What must be prepared once, per tenant, is the identity provider: an
initialized Logto tenant with its roles and its owner account. The
secrets are listed in
.github/workflows/README.md.Notes
Three changes reach the application code; two of them change what
production builds do:
App.vuemountedPiniaColadaProdDevtools, which ships the panel in every build. It nowmounts
PiniaColadaDevtools, which is dev-only, so the panel no longerappears on qa or production.
mid-visit,
stores/login.tssaves the page the user is on beforeredirecting to Logto, so they come back there instead of on the
dashboard.
VITE_E2Eflag, set only for test builds, disablesbackground refetching and the devtools panel, because both interfere
with the tests. It has no effect on dev, qa or production builds.
Also adds a cheap authorization check to the existing pipeline: it fails
when an endpoint ships without a declared permission, or when the test
fixture names a role no configuration defines.