Repository navigation
fix(model): open Metal devices on macOS before 15 (candle residency-set guard) - #48
Merged
Merged
Conversation
candle 0.11 creates a residency set with each Metal device, and residency sets arrived in macOS 15, so opening a device panics on macOS 14 and earlier. This test opens one and runs a kernel; CI's macos-14 runner exercises it. author: Tin Dang
candle 0.11.0 creates a residency set with every Metal device, and MTLResidencySetDescriptor only exists from macOS 15, so on macOS 14 and earlier opening a device panicked: the daemon's model thread died and every prompt was ranked by keywords alone. candle-metal-kernels 0.11.0 is vendored with one guard (no descriptor class, no residency set), which candle already handles, and applied through [patch.crates-io]. PATCHED.md says when to drop it. author: Tin Dang
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On macOS 14 and earlier, laya-codex has never been able to open a Metal device, so the Laya model never loaded there.
Cause: candle 0.11.0 (the latest release, used since v0.1.0) creates a residency set every time it opens a Metal device. Residency sets arrived in macOS 15, and on older systems
MTLResidencySetDescriptor::new()panics because the class doesn't exist. That panic happens on the daemon's model-loading thread, so on those Macs the model silently never loads and every prompt ranks with keywords only. CI didn't catch it because no test opened a Metal device until the kernel work in #45, whose macos-14 job hits exactly this panic.Fix: vendor candle-metal-kernels 0.11.0 (MIT OR Apache-2.0) with a one-line guard, applied through
[patch.crates-io]. When the class is missing,ResidencySet::newreturns an empty set. candle already supports that path:raw()isNoneand nothing is added to the command queue.vendor/candle-metal-kernels/PATCHED.mdrecords the change and when to drop it.Proof on the macOS that matters:
crates/laya-model/tests/metal_device.rs: open a Metal device and run a kernel). It is expected to fail on CI's macos-14 runner.Locally on macOS 15: fmt, clippy
-D warnings, workspace tests and the new test all pass.