Repository navigation
Conversation
Adds integration-tests/atx-transform-server/src/tests/atxLifecycle.test.ts covering startTransform failures, stopping jobs, listJobs and router input validation. Tests only vary inputs a real client could send; direct FES calls are used only for setup and cleanup the LSP cannot do. - atxTestHarness.ts: shared LSP session (sign-in, profile, request timeouts), job tracking, orphaned-job detection, and stop+delete cleanup. Each suite run uses its own workspace so cleanup never touches jobs from other runs in the same account. - lspClient.ts: when LSP_LOG_FILE is set, write the server's stdout/stderr and window/logMessage output to that file (previously discarded, which hid the FES total-jobs quota error behind "StartTransform workflow failed"). - package.json: declare adm-zip and @amazon/elastic-gumby-frontend-client, which the tests previously resolved only through workspace hoisting. The suite deletes every job it creates. The existing E2E suite does not, which exhausted the account's FES total-jobs quota between 2026-08-25 and 2026-10-07.
|
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
The suite's workspace can also receive jobs from other clients: the scheduled E2E suite picks the first workspace listOrCreateWorkspace returns, which can be this suite's temporary workspace. Cleanup treated those jobs as orphans and stopped and deleted them, which broke the other run. Each session now tags the jobs it creates with a run ID. Only tagged jobs can be orphans; untagged jobs are logged and left running, and the workspace is kept while another client still has jobs in it.
Adds integration-tests/atx-transform-server/src/tests/atxOutput.test.ts. - Upload contents (fast): start a job, open the uploaded zip, and check requirement.json, transformation-preferences.json and every source file (MD5 and bytes) against the request, for two sets of transform flags. - One real transform (slow): chat round trip while the agent plans; poll to COMPLETED with SolutionRootPath the way the IDE does, checking that each step's changes apply to the local solution, plan checkpoints and worklogs; download every output artifact and check the transformed .csproj files target net8.0; page in older worklogs. The slow job runs on a temp copy of the fixture, because polling with SolutionRootPath writes into the solution folder. Two current behaviors are recorded rather than asserted as correct: - A step can show COMPLETED one poll before its changes can be downloaded, so that poll reports DiffApplyFailed. The test allows this as long as the step is applied on a later poll. - On COMPLETED, the LSP extracts the first CUSTOMER_OUTPUT artifact (Transformation_Report.html) instead of TransformedSource.zip, so ArtifactPath is null. The test checks TransformedSource.zip directly. atxTestHarness.ts (additive): copyFixture, token refresh from Secrets Manager for runs longer than the token lifetime, extra getTransformInfo inputs in getStatus, and StartTransformRequest overrides in startTransform.
The agent names the transformed-source artifact itself (seen: TransformedSource.zip, BobsBookstoreClassic_Migrated.zip), so pick the one .zip in listArtifacts instead of matching a fixed name. Also use the session's run-tagged job names and keep the workspace while other clients still have jobs in it.
cslong
force-pushed
the
dev/longachr/atx-output-integ-tests
branch
from
October 9, 2026 21:24
4f5eb10 to
5ec2e48
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The ATX .NET integration tests check only that a transform reaches COMPLETED. They don't check what the LSP sends to the service, or what it does with the results. In particular, they don't cover:
The existing E2E test also polls without
SolutionRootPath, which the IDE always sends. Because of that, the LSP's apply step fails on every poll and reportsDiffApplyFailed: true, and the test passes anyway.Solution
This PR adds
atxOutput.test.ts. Same ground rules as the lifecycle suite: there are no mocks, every behavior is triggered by an input a real client could send, and every job and workspace the suite creates is deleted.requirement.jsonhas the right entry, solution and pm flags from the request. Everysource file is listed with its e same bytes.`transformation-preferences.jsoned on. Runs once with theFINAL_RESPONSE. If it answerss with the agent's message. Also checkslistMessagespaginatioing, andbatchGetMessages.lirtifactfor every output; eachdownloaded size matches the lis extracts, and every.csprojin it targets net8.0.Testing
Ran locally against the Transfo- Upload tests: passed in e- Slow tests: passed in fule were recorded. Final localverification run: .- Stalled job: one run hit failure. The agent answered thekickoff with "There was an inte the agent", and the job stayedin PLANNING. The new chat checkes the 30-minute planning guard.
License
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.