Skip to content

test(e2e): pull container images on-demand for filtered test runs - #1041

Open
Nachiket-Roy wants to merge 1 commit into
urunc-dev:mainfrom
Nachiket-Roy:fix/test-image
Open

Nachiket-Roy wants to merge 1 commit into
urunc-dev:mainfrom
Nachiket-Roy:fix/test-image

Conversation

@Nachiket-Roy

@Nachiket-Roy Nachiket-Roy commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Description

The end-to-end test suite previously pulled all declared container images in BeforeAll regardless of test filters. When executing a filtered subset of tests (such as make test_nerdctl_Spt or specific CI/CD jobs), this incurred unnecessary network overhead, bandwidth usage, and pull flakiness.
The test framework now:

  • Pulls images on-demand: Images and sidecars are pulled only when a matching spec executes (before pod or container creation).
  • Prevents duplicate pulls: Uses a simple thread-safe in-memory map to ensure each image is pulled at most once per test run.
  • Cleans up pulled images: DeferCleanup only removes images that were actually downloaded during the test run.
  • Reuses existing abstractions: Leverages tool.Name() ("ctr", "crictl", "docker", "nerdctl") without introducing extra types or interface methods.

Related issues

How was this tested?

  1. Build and Compilation

    go build ./... && go test -c ./tests/e2e -o /dev/null

    The entire codebase and e2e test harness compile cleanly without errors.

  2. Full Unit Test Suite

    make unittest

    All unit tests across pkg/network, pkg/unikontainers, pkg/unikontainers/hypervisors, pkg/unikontainers/initrd, pkg/unikontainers/unikernels, and internal/metrics passed with zero errors.

  3. Filtered Dry‑Run Verification

    go test ./tests/e2e -v -run TestE2E --ginkgo.focus="Nerdctl.*Spt" --ginkgo.dry-run

    Output:

    Will run 4 of 75 specs
    SSSSSSSSSSSSSSSSSSS•SSSSSSS•••SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS
    Ran 4 of 75 Specs in 0.002 seconds
    SUCCESS! -- 4 Passed | 0 Failed | 0 Pending | 71 Skipped
    

    Only the 4 matching specs were targeted, and no unneeded images from the remaining 71 test cases were queued or pulled.

LLM usage

Gemini 3.8

Checklist

  • I have read the contribution guide.
  • The linter passes locally (make lint).
  • The e2e tests of at least one tool pass locally (make test_ctr, make test_nerdctl, make test_docker, make test_crictl).
  • If LLMs were used: I have read the llm policy.

@netlify

netlify Bot commented Sep 9, 2026

Copy link
Copy Markdown

Deploy Preview for urunc ready!

Name Link
🔨 Latest commit c4fa34e
🔍 Latest deploy log https://app.netlify.com/projects/urunc/deploys/6aad1b3c2543920008452bee
😎 Deploy Preview https://deploy-preview-1041--urunc.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@cmainas cmainas left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @Nachiket-Roy ,

there are multiple issues with this proposal:

  • The ToolType replicates the Name method.
  • The issue did not mention to retain images
  • I can not really understand the purpose of the lock
  • There is no reason to check if an image exists, An existing image will not get pulled again.

This implementation is unnecessary overcomplicated.

The end-to-end test suite previously pulled all declared container images
in BeforeAll regardless of test filters. When executing a filtered subset
of tests (such as make test_nerdctl_Spt or ARM64 CI/CD jobs), this incurred
substantial network latency, bandwidth usage, and pull flakiness.

Transition to on-demand image pulling by pulling container images only when
a test spec is executed:
- Implement ensureTestImages to lazily pull container images and sidecars
  before container or pod creation.
- Track pulled images in-memory to prevent duplicate pulls within the run.
- Clean up only the images pulled during the test run in DeferCleanup.

Fixes urunc-dev#1037

Signed-off-by: Nachiket Roy <nachiket.roy.2@gmail.com>
@Nachiket-Roy

Copy link
Copy Markdown
Contributor Author

Thanks for the review. I have resolved them and simplified the PR

  • The ToolType replicates the Name method.

Reverted

  • The issue did not mention to retain images

sure removed it

  • I can not really understand the purpose of the lock

Added to be concurrent safe but that is already handled so droped

  • There is no reason to check if an image exists, An existing image will not get pulled again.

Dropped

please take a look again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PUll only the images to test

2 participants