Skip to content

fix(cuda-macros): obey #[inline] on generic device functions - #1472

Draft
fxeqxmulfx wants to merge 1 commit into
NVIDIA:mainfrom
fxeqxmulfx:fix/1471-generic-device-inline-policy
Draft

fxeqxmulfx wants to merge 1 commit into
NVIDIA:mainfrom
fxeqxmulfx:fix/1471-generic-device-inline-policy

Conversation

@fxeqxmulfx

@fxeqxmulfx fxeqxmulfx commented Oct 10, 2026 •

Copy link
Copy Markdown

Summary

#[device] added #[inline(never)] before the other attributes of a generic function. rustc uses only the first inline attribute. Because of this, rustc did not use an #[inline(always)] or #[inline] attribute on the function. rustc also gave an unused attribute warning. Now #[device] adds #[inline(never)] only if there is no inline attribute on the function.

Fixes #1471

Before → after

#[device]
#[inline(always)]
fn scale<T: Copy + core::ops::Mul<Output = T>>(value: T, factor: T) -> T {
    value * factor
}

#[kernel]
pub fn scale_all(input: &[f32], mut output: DisjointSlice<f32>) {
    let idx = thread::index_1d();
    let i = idx.get();
    if let Some(out) = output.get_mut(idx) {
        *out = scale(input[i], 2.0);
    }
}
main This PR
inline attributes of {DEVICE_PREFIX}scale #[inline(never)] (from #[device]), #[inline(always)] #[inline(always)]
rustc warning: unused attribute for #[inline(always)]: "it will become a hard error in a future release" No warning
Device LLVM IR before opt scale_all calls scalefEB2_ No scalefEB2_: scale_all contains the code of scale
PTX .entry scale_all and .visible .func scalefEB2_ .entry scale_all only

Changes

  • cuda-macros/src/device.rs: #[device] adds #[inline(never)] to a generic function only if the function has no inline attribute. The check uses attr_path_ends_with. #[kernel] uses the same function for the attributes of generic kernels. rustc expands cfg_attr before it calls the macro. Because of this, the check also finds an inline attribute in cfg_attr.
  • The expansion is now in device_function_tokens, as for generic kernels in generic_kernel_no_instantiation_tokens. Unit tests can examine it.
  • Regression tests: the unit test generic_device_function_keeps_the_inline_attribute_of_the_user (#[inline], #[inline(always)], #[inline(never)]), and the trybuild pass test tests/pass/device_inline_policy.rs with #![deny(unused_attributes)]. Both fail when #[device] always adds #[inline(never)], as on main.
  • Control tests: generic_device_function_is_inline_never_by_default and non_generic_device_function_gets_no_inline_attribute.
  • Negative test: other_attributes_do_not_replace_the_default (#[cold], #[doc = "inline"]).
  • standalone_device_fn: the comment about #[inline(never)] now includes "by default".

Testing

  • just -f cuda-oxide/Justfile check passes (the local mirror of CI: fmt, clippy, tests, guards, docs)
  • cargo oxide run <example> passes, or cuda-oxide/scripts/smoketest.sh -o '^<example>$'
  • New example added (if applicable)

cargo oxide run standalone_device_fn --arch sm_80 gives SUCCESS: 18/18 tests passed.

On an RTX 3050 (sm_86), the example above gives correct results on main and with this change. The table shows the output of the two builds.

Checklist

  • All commits signed off (git commit -s)
  • SPDX headers on new source files

🤖 Generated with Claude Code

@copy-pr-bot

copy-pr-bot Bot commented Oct 10, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@fxeqxmulfx fxeqxmulfx closed this Oct 10, 2026
@fxeqxmulfx fxeqxmulfx reopened this Oct 10, 2026
@fxeqxmulfx fxeqxmulfx closed this Oct 10, 2026
@fxeqxmulfx fxeqxmulfx reopened this Oct 11, 2026
@fxeqxmulfx
fxeqxmulfx force-pushed the fix/1471-generic-device-inline-policy branch from 4c12128 to 74ce9f6 Compare October 11, 2026 17:51
`#[device]` added `#[inline(never)]` before the other attributes of a
generic device function. rustc uses only the first `inline` attribute.
Because of this, rustc did not use an `#[inline(always)]` or
`#[inline]` attribute on the function.

Add `#[inline(never)]` only if there is no `inline` attribute on the
function. `#[kernel]` uses the same check for the attributes of generic
kernels. rustc expands `cfg_attr` before it calls the macro. Because of
this, the check also finds an `inline` attribute in `cfg_attr`.

Move the expansion to `device_function_tokens`, as for generic kernels,
so that unit tests can examine it.

Fixes NVIDIA#1471

Signed-off-by: Mikhail Borisov <mikhail.boriso2014@gmail.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@fxeqxmulfx
fxeqxmulfx force-pushed the fix/1471-generic-device-inline-policy branch from 74ce9f6 to a146283 Compare October 11, 2026 20:50
@fxeqxmulfx
fxeqxmulfx marked this pull request as draft October 12, 2026 08:53
@nihalpasham nihalpasham added the cuda-oxide SIMT programming model: rustc backend, cargo oxide, cuda-device, cuda-host, book label Oct 15, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cuda-oxide SIMT programming model: rustc backend, cargo oxide, cuda-device, cuda-host, book

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG]: #[device] does not obey #[inline] on a generic function

3 participants