Repository navigation
fix(vm): boot a digest-pinned image from the cache without a registry call - #4582
Open
balajinvda wants to merge 1 commit into
Open
balajinvda wants to merge 1 commit into
balajinvda wants to merge 1 commit into
Conversation
… call A registry-sourced sandbox boot authenticated with the registry, and for a tag resolved the manifest, before it looked at the root disk cache. It did so on every boot, even when the image reference carries a digest and the root disk for that digest is already cached. Each boot then waits on the registry, and a registry outage stops every boot of an image the host already has. A digest names the image's content, and since NVIDIA#4342 a cached root disk is published only after its manifest was verified against that digest. On a cache hit for a digest-pinned reference the registry cannot add anything, so return the hit before authenticating. Authentication and the pull stay on the miss path, and a tag still resolves through the registry. The hit emits the same Pulling, CacheHit and Pulled events as before. Behaviour change: a host whose registry credentials were revoked still boots digest-pinned images it has already cached, as a kubelet does with IfNotPresent and a digest-pinned image. Test: bootstrap_registry_cache_hit_for_a_digest_pin_makes_no_registry_request covers a miss (authenticates and fetches), a cached digest pin (no request at all) and a tag for the same content (still resolved). It fails without this change with ["GET /v2/"]. Closes NVIDIA#4581 Signed-off-by: Balaji Ganesan <bganesan@nvidia.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A VM sandbox boot from the registry authenticated with the registry before it looked at the root disk cache, on every boot, even for a digest-pinned image whose root disk is cached. This returns the cache hit first for digest-pinned references, so such a boot makes no registry request and keeps working while the registry is unreachable.
Related Issue
Closes #4581
Changes
ensure_cached_registry_rootfs_image: when the reference carries a digest andimage_cache_rootfs_image(bootstrap_image_cache_identity(digest))exists, return the cache hit beforeclient.auth. Authentication and the pull stay on the miss path; tags still resolve through the registry.Pulling,CacheHit,Pulled) move intopublish_registry_cache_hit, shared by both hit paths, so a digest-pinned hit reports exactly as before.IfNotPresentand a digest-pinned image.e1f3c82c) on arm64 hosts against a remote registry with the image cached: registry authentication and resolution were about 1.9 s (p50, 96 boots) of a 2.5 s boot (numbers in VM driver authenticates with the registry on every boot, even for a cached digest-pinned image #4581). The same code path authenticates on every boot in v0.1.4 (5a7728c3) and onmainat909f336b6, which this change is based on; since fix(vm): bind cached registry images to verified manifest digests #4342 a digest pin skips only the manifest request there, not the authentication.Testing
cargo fmt --check,cargo clippy -p openshell-driver-vm --all-targets -- -D warnings,cargo test -p openshell-driver-vmbootstrap_registry_cache_hit_for_a_digest_pin_makes_no_registry_request(stub registry): a miss authenticates and fetches; a cached digest pin makes no request; a tag for the same content still resolves. Without the change it fails with["GET /v2/"].cargo test -p openshell-driver-vmalso failsstopping_a_sandbox_lets_the_supervisor_drain_before_closing_livenesson the test machine; it fails the same way on unpatchedmain, so it is not related to this change.Checklist