Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
2531797
CTK 13.4 prerelease integration (#2437)
mdboom Jul 28, 2026
7a8463b
13.4 prerelease: Fix release automation issues (#2445)
mdboom Jul 29, 2026
83023b6
Revert CTK installation workarounds
mdboom Aug 21, 2026
8f75d26
Merge remote-tracking branch 'upstream/main' into 13.4.x
mdboom Sep 9, 2026
9d762dd
Add Windows-on-ARM support
mdboom Sep 9, 2026
3d204cb
git merge --squash ctk-next-public-main-2026-09-04+1303-merge; preser…
rwgk Sep 4, 2026
a140f47
Update release notes and versions
mdboom Sep 9, 2026
1464def
Remove obsolete cuda_bindings test_helper
mdboom Sep 9, 2026
7424b41
Fix regression in NVML docstring
mdboom Sep 9, 2026
7cfdad7
Fix Windows on ARM build
mdboom Sep 9, 2026
c221b55
Fix up versions in release notes a bit
mdboom Sep 9, 2026
554bf2b
Update pixi configs
mdboom Sep 9, 2026
a6f6adc
Exclude Python 3.10 on Windows-on-ARM
mdboom Sep 9, 2026
4581b96
Remove cupti dlls
mdboom Sep 9, 2026
6a49756
Skip flaky test mentioned in #2790
mdboom Sep 9, 2026
0dda534
Fix version in release notes
mdboom Sep 9, 2026
51ad4ac
Address agent feedback #3
mdboom Sep 9, 2026
0b233a4
Address agent feedback #4: Update unsupported_before calls to use Non…
mdboom Sep 9, 2026
4936dcd
Don't include pre-releases in the toctree
mdboom Sep 9, 2026
c10db05
Revert test skip
mdboom Sep 9, 2026
946f067
test(cuda.core): serialize thread-unsafe IPC error tests (#2785)
rluo8 Sep 9, 2026
4732fec
Revert "cuda.core: refresh frozen CUresult explanation table for CUDA…
rwgk Sep 9, 2026
52c4c14
Validate that the ctk restored correctly
mdboom Sep 9, 2026
e253173
Add comment: Do not update it past CUDA Toolkit v13.1.1
rwgk Sep 9, 2026
901323d
Merge remote-tracking branch 'rwgk/refreeze_enum_tables' into 13.4.x
mdboom Sep 9, 2026
452b9a1
Remove erroneous validation
mdboom Sep 9, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 0 additions & 15 deletions .github/RELEASE-core.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,21 +89,6 @@ changes, so they are only permitted at a major-version boundary per the

---

## Refresh frozen fallback explanation tables

When this release adds support for a new CUDA Toolkit version, check
whether the CTK release added any new `CUresult` or `cudaError_t` codes.
If so, update the frozen fallback tables so error messages stay
informative for consumers on older `cuda-bindings` versions:

- `cuda_core/cuda/core/_utils/driver_cu_result_explanations_frozen.py`
- `cuda_core/cuda/core/_utils/runtime_cuda_error_explanations_frozen.py`

The corresponding tests in `test_utils_enum_explanations_helpers.py`
will fail if an entry is missing.

---

## Finalize the doc update, including release notes

Review every PR included in the release. For each one, check whether new
Expand Down
22 changes: 9 additions & 13 deletions .github/actions/fetch_ctk/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ runs:
function extract() {
tar -xvf $1 -C $CACHE_TMP_DIR --strip-components=1
}
elif [[ "${{ inputs.host-platform }}" == "win-64" ]]; then
elif [[ "${{ inputs.host-platform }}" == win* ]]; then
function extract() {
_TEMP_DIR_=$(mktemp -d)
unzip $1 -d $_TEMP_DIR_
Expand Down Expand Up @@ -120,8 +120,12 @@ runs:
# Note: try to escape | and > ...
tar -czvf ${CTK_CACHE_FILENAME} ${CACHE_TMP_DIR}

# "Move" files from temp dir to CUDA_PATH
CUDA_PATH="./cuda_toolkit"
# Populate the final CUDA_PATH directly (never stage through ./cuda_toolkit
# unconditionally — a second call with a different cuda-path would leave
# symlinks from a prerelease CTK at ./cuda_toolkit/include etc., causing
# "cp: cannot overwrite non-directory" failures on the next restore).
CUDA_PATH="${{ inputs.cuda-path }}"
rm -rf $CUDA_PATH
mkdir -p $CUDA_PATH
# Unfortunately we cannot use "rsync -av $CACHE_TMP_DIR/ $CUDA_PATH" because
# not all runners have rsync pre-installed (or even installable, such as
Expand All @@ -144,22 +148,14 @@ runs:
run: |
ls -l
CACHE_TMP_DIR="./cache_tmp_dir"
CUDA_PATH="./cuda_toolkit"
CUDA_PATH="${{ inputs.cuda-path }}"
rm -rf $CUDA_PATH
mkdir -p $CUDA_PATH
tar -xzvf $CTK_CACHE_FILENAME
# Can't use rsync here, see above
cp -r $CACHE_TMP_DIR/* $CUDA_PATH
rm -rf $CACHE_TMP_DIR $CTK_CACHE_FILENAME
ls -l $CUDA_PATH
if [ ! -d "$CUDA_PATH/include" ]; then
exit 1
fi

- name: Move CTK to the specified location
if: ${{ inputs.cuda-path != './cuda_toolkit' }}
shell: bash --noprofile --norc -xeuo pipefail {0}
run: |
mv ./cuda_toolkit ${{ inputs.cuda-path }}

- name: Set output environment variables
shell: bash --noprofile --norc -xeuo pipefail {0}
Expand Down
54 changes: 39 additions & 15 deletions .github/workflows/build-wheel.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@ on:
required: false
type: string
default: ""
single-cuda-major:
description: "Build wheels for only the current CUDA major; skip the prior-major build and wheel merge"
required: false
type: boolean
default: false

defaults:
run:
Expand Down Expand Up @@ -51,10 +56,16 @@ jobs:
- "3.14t"
- "3.15"
- "3.15t"
exclude:
# CPython 3.10 has no official Windows ARM64 build (neither
# nuget-cpython nor actions/setup-python's manifest carries one),
# so it cannot be built or tested on win-arm64.
- python-version: ${{ (inputs.host-platform == 'win-arm64' && '3.10') || '' }}
name: py${{ matrix.python-version }}
runs-on: ${{ (inputs.host-platform == 'linux-64' && 'linux-amd64-cpu8') ||
(inputs.host-platform == 'linux-aarch64' && 'linux-arm64-cpu8') ||
(inputs.host-platform == 'win-64' && 'windows-2022') }}
(inputs.host-platform == 'win-64' && 'windows-2022') ||
(inputs.host-platform == 'win-arm64' && 'windows-11-arm') }}
steps:
- name: Checkout ${{ github.event.repository.name }}
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
Expand Down Expand Up @@ -87,7 +98,7 @@ jobs:
uses: nv-gha-runners/setup-proxy-cache@main
continue-on-error: true
# Skip cache on GitHub-hosted Windows runners.
if: ${{ inputs.host-platform != 'win-64' }}
if: ${{ !startsWith(inputs.host-platform, 'win') }}
with:
enable-apt: true

Expand All @@ -98,22 +109,26 @@ jobs:
# WAR: setup-python is not relocatable, and cibuildwheel hard-wires to 3.12...
# see https://github.com/actions/setup-python/issues/871
python-version: "3.12"
architecture: ${{ ((inputs.host-platform == 'linux-aarch64' || inputs.host-platform == 'win-arm64') && 'arm64') || 'x64' }}

- name: Set up MSVC
if: ${{ startsWith(inputs.host-platform, 'win') && (env.BUILD_BINDINGS == 'true' || env.BUILD_CORE == 'true' || env.TEST_BINDINGS == 'true' || env.TEST_CORE == 'true') }}
uses: step-security/msvc-dev-cmd@22c98154b708dbd743e6f27a933cf6ceba3305c4 # v1.13.1
with:
arch: ${{ (inputs.host-platform == 'win-arm64' && 'arm64') || 'x64' }}

- name: Set up yq
# GitHub made an unprofessional decision to not provide it in their Windows VMs,
# see https://github.com/actions/runner-images/issues/7443.
if: ${{ startsWith(inputs.host-platform, 'win') && env.BUILD_CORE == 'true' }}
env:
YQ_VERSION: v4.52.5
YQ_SHA256: 47594981f3848a4b4447494adeca9555f908f7cf0a89c4da3fd0243a4631da1c
YQ_ARCH: ${{ (inputs.host-platform == 'win-arm64' && 'arm64') || 'amd64' }}
YQ_SHA256: ${{ (inputs.host-platform == 'win-arm64' && '236867affa7f18701d4c763cf16b6df962cf4f7e89a8570a5954cf94a38f41c7') || '47594981f3848a4b4447494adeca9555f908f7cf0a89c4da3fd0243a4631da1c' }}
YQ_DIR: yq
shell: pwsh -command ". '{0}'"
run: |
$yqUrl = "https://github.com/mikefarah/yq/releases/download/${env:YQ_VERSION}/yq_windows_amd64.exe"
$yqUrl = "https://github.com/mikefarah/yq/releases/download/${env:YQ_VERSION}/yq_windows_${env:YQ_ARCH}.exe"
mkdir -Force -ErrorAction SilentlyContinue "${env:YQ_DIR}" | Out-Null
Invoke-WebRequest -UseBasicParsing -OutFile "${env:YQ_DIR}/yq.exe" -Uri "$yqUrl"
$hash = (Get-FileHash -Algorithm SHA256 "${env:YQ_DIR}/yq.exe").Hash.ToLower()
Expand Down Expand Up @@ -253,7 +268,7 @@ jobs:
echo "ok!"

- name: Report sccache stats (cuda.bindings)
if: ${{ env.BUILD_BINDINGS == 'true' && inputs.host-platform != 'win-64' }}
if: ${{ env.BUILD_BINDINGS == 'true' && !startsWith(inputs.host-platform, 'win') }}
uses: ./.github/actions/sccache-summary
with:
json-file: sccache_bindings.json
Expand Down Expand Up @@ -362,7 +377,7 @@ jobs:
echo "ok!"

- name: Report sccache stats (cuda.core)
if: ${{ env.BUILD_CORE == 'true' && inputs.host-platform != 'win-64' }}
if: ${{ env.BUILD_CORE == 'true' && !startsWith(inputs.host-platform, 'win') }}
uses: ./.github/actions/sccache-summary
with:
json-file: sccache_core.json
Expand Down Expand Up @@ -524,7 +539,7 @@ jobs:

# Note: This overwrites CUDA_PATH etc
- name: Set up mini CTK
if: ${{ env.BUILD_CORE == 'true' || env.TEST_CORE == 'true' }}
if: ${{ !inputs.single-cuda-major && (env.BUILD_CORE == 'true' || env.TEST_CORE == 'true') }}
uses: ./.github/actions/fetch_ctk
continue-on-error: false
with:
Expand All @@ -533,13 +548,13 @@ jobs:
cuda-path: "./cuda_toolkit_prev"

- name: Build cuda.core test binaries
if: ${{ env.TEST_CORE == 'true' }}
if: ${{ !inputs.single-cuda-major && env.TEST_CORE == 'true' }}
run: |
nvcc --version
python "${{ env.CUDA_CORE_TEST_BINARIES_DIR }}/build_test_binaries.py"

- name: Upload cuda.core test binaries
if: ${{ env.TEST_CORE == 'true' }}
if: ${{ !inputs.single-cuda-major && env.TEST_CORE == 'true' }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: ${{ env.CUDA_CORE_ARTIFACT_NAME }}-test-binaries
Expand All @@ -550,7 +565,7 @@ jobs:
if-no-files-found: error

- name: Download cuda.bindings build artifacts from the prior branch
if: ${{ env.BUILD_CORE == 'true' }}
if: ${{ !inputs.single-cuda-major && env.BUILD_CORE == 'true' }}
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
Expand Down Expand Up @@ -586,7 +601,7 @@ jobs:
rmdir "${OLD_ARTIFACT_DIR}"

- name: Constrain previous cuda.core to the downloaded cuda.bindings wheel
if: ${{ env.BUILD_CORE == 'true' }}
if: ${{ !inputs.single-cuda-major && env.BUILD_CORE == 'true' }}
run: |
pathfinder_wheels=(cuda_pathfinder/cuda_pathfinder-*.whl)
bindings_wheels=(cuda_bindings/dist-prev/cuda_bindings-"${BUILD_PREV_CUDA_MAJOR}".*.whl)
Expand All @@ -608,7 +623,7 @@ jobs:
} | tee wheel-constraints/cuda-core-prev.txt

- name: Build cuda.core wheel
if: ${{ env.BUILD_CORE == 'true' }}
if: ${{ !inputs.single-cuda-major && env.BUILD_CORE == 'true' }}
uses: pypa/cibuildwheel@1828c10ab37f080699c7b81cea34097c684a7074 # v4.2.0
with:
package-dir: ./cuda_core/
Expand Down Expand Up @@ -656,15 +671,15 @@ jobs:
echo "ok!"

- name: Report sccache stats (cuda.core prev)
if: ${{ env.BUILD_CORE == 'true' && inputs.host-platform != 'win-64' }}
if: ${{ !inputs.single-cuda-major && env.BUILD_CORE == 'true' && !startsWith(inputs.host-platform, 'win') }}
uses: ./.github/actions/sccache-summary
with:
json-file: sccache_core_prev.json
label: "cuda.core (prev CTK)"
build-step: "Build cuda.core wheel"

- name: List the cuda.core artifacts directory and rename
if: ${{ env.BUILD_CORE == 'true' }}
if: ${{ !inputs.single-cuda-major && env.BUILD_CORE == 'true' }}
run: |
if [[ "${{ inputs.host-platform }}" == win* ]]; then
export CHOWN=chown
Expand All @@ -688,14 +703,23 @@ jobs:
ls -lahR ${{ env.CUDA_CORE_ARTIFACTS_DIR }}

- name: Merge cuda.core wheels
if: ${{ env.BUILD_CORE == 'true' }}
if: ${{ !inputs.single-cuda-major && env.BUILD_CORE == 'true' }}
run: |
pip install wheel
python ci/tools/merge_cuda_core_wheels.py \
"${{ env.CUDA_CORE_ARTIFACTS_DIR }}"/cu"${BUILD_CUDA_MAJOR}"/cuda_core*.whl \
"${{ env.CUDA_CORE_ARTIFACTS_DIR }}"/cu"${BUILD_PREV_CUDA_MAJOR}"/cuda_core*.whl \
--output-dir "${{ env.CUDA_CORE_ARTIFACTS_DIR }}"

- name: Finalize single-major cuda.core wheel
if: ${{ inputs.single-cuda-major && env.BUILD_CORE == 'true' }}
run: |
for wheel in "${{ env.CUDA_CORE_ARTIFACTS_DIR }}"/cu"${BUILD_CUDA_MAJOR}"/*.cu"${BUILD_CUDA_MAJOR}".whl; do
base_name=$(basename "${wheel}" ".cu${BUILD_CUDA_MAJOR}.whl")
mv "${wheel}" "${{ env.CUDA_CORE_ARTIFACTS_DIR }}/${base_name}.whl"
done
ls -lahR "${{ env.CUDA_CORE_ARTIFACTS_DIR }}"

- name: Check cuda.core wheel
if: ${{ env.BUILD_CORE == 'true' }}
run: |
Expand Down
62 changes: 62 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ jobs:
outputs:
CUDA_BUILD_VER: ${{ steps.get-vars.outputs.cuda_build_ver }}
CUDA_PREV_BUILD_VER: ${{ steps.get-vars.outputs.cuda_prev_build_ver }}
WINDOWS_ARM64_SUPPORTED: ${{ steps.get-vars.outputs.windows_arm64_supported }}
WINDOWS_ARM64_SINGLE_CUDA_MAJOR: ${{ steps.get-vars.outputs.windows_arm64_single_cuda_major }}
steps:
- name: Checkout repository
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
Expand All @@ -47,6 +49,33 @@ jobs:
cuda_prev_build_ver=$(yq '.cuda.prev_build.version' ci/versions.yml)
echo "cuda_prev_build_ver=$cuda_prev_build_ver" >> $GITHUB_OUTPUT

# Windows ARM64 is available starting with CUDA 13.4.
if [[ "$cuda_build_ver" =~ ^([0-9]+)\.([0-9]+)(\.|$) ]]; then
cuda_build_major="${BASH_REMATCH[1]}"
cuda_build_minor="${BASH_REMATCH[2]}"
else
echo "Invalid CUDA build version: $cuda_build_ver" >&2
exit 1
fi
if (( cuda_build_major > 13 || (cuda_build_major == 13 && cuda_build_minor >= 4) )); then
windows_arm64_supported=true
else
windows_arm64_supported=false
fi
echo "windows_arm64_supported=$windows_arm64_supported" >> $GITHUB_OUTPUT

# No CUDA 13 windows-arm64 toolkit exists for a major other than the
# current one (windows-arm64 support started mid-way through the 13.x
# series), so cuda.core can only be built against a single CUDA major
# while the build major is still 13. Once the build major advances to
# 14, a CUDA 13 windows-arm64 toolkit will exist as the prior major.
if [[ "$cuda_build_major" == "13" ]]; then
windows_arm64_single_cuda_major=true
else
windows_arm64_single_cuda_major=false
fi
echo "windows_arm64_single_cuda_major=$windows_arm64_single_cuda_major" >> $GITHUB_OUTPUT
Comment on lines +52 to +77

@leofang leofang Sep 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only use 1 CUDA major.minor to build in the main branch. Now that we bump the build floor I think this can be dropped and simplified.


should-skip:
if: ${{ github.repository_owner == 'nvidia' }}
runs-on: ubuntu-latest
Expand Down Expand Up @@ -414,6 +443,31 @@ jobs:
prev-cuda-version: ${{ needs.ci-vars.outputs.CUDA_PREV_BUILD_VER }}
workplan: ${{ needs.detect-changes.outputs.workplan }}

# Windows ARM64 is available starting with CUDA 13.4. Build only the current
# CUDA major because no prior-major toolkit or wheel artifacts exist.
build-windows-arm64:
needs:
- ci-vars
- should-skip
- detect-changes
name: Build win-arm64, CUDA ${{ needs.ci-vars.outputs.CUDA_BUILD_VER }}
if: ${{ github.repository_owner == 'nvidia' &&
!fromJSON(needs.should-skip.outputs.skip) &&
!fromJSON(needs.should-skip.outputs.doc-only) &&
needs.ci-vars.outputs.WINDOWS_ARM64_SUPPORTED == 'true' &&
fromJSON(needs.detect-changes.outputs.workplan).jobs.platforms.windows }}
permissions:
actions: read
contents: read
secrets: inherit
uses: ./.github/workflows/build-wheel.yml
with:
host-platform: win-arm64
cuda-version: ${{ needs.ci-vars.outputs.CUDA_BUILD_VER }}
prev-cuda-version: ${{ needs.ci-vars.outputs.CUDA_PREV_BUILD_VER }}
workplan: ${{ needs.detect-changes.outputs.workplan }}
single-cuda-major: ${{ needs.ci-vars.outputs.WINDOWS_ARM64_SINGLE_CUDA_MAJOR == 'true' }}

# NOTE: test-sdist jobs are split by platform (mirroring build-* and test-wheel-*)
# so platform-specific sources (e.g. cuda_bindings/*_windows.pyx selected by
# build_hooks.py) are exercised on their target OS. Keep these job definitions
Expand Down Expand Up @@ -613,6 +667,7 @@ jobs:
- build-linux-64
- build-linux-aarch64
- build-windows
- build-windows-arm64
- test-sdist-linux
- test-sdist-windows
- test-linux-64
Expand Down Expand Up @@ -644,6 +699,7 @@ jobs:
doc_only="${{ needs.should-skip.outputs.doc-only }}"
linux_selected="${{ needs.detect-changes.outputs.workplan && fromJSON(needs.detect-changes.outputs.workplan).jobs.platforms.linux || false }}"
windows_selected="${{ needs.detect-changes.outputs.workplan && fromJSON(needs.detect-changes.outputs.workplan).jobs.platforms.windows || false }}"
windows_arm64_supported="${{ needs.ci-vars.outputs.WINDOWS_ARM64_SUPPORTED }}"
build_selected="${{ needs.detect-changes.outputs.workplan && fromJSON(needs.detect-changes.outputs.workplan).jobs.sdist_tests || false }}"
run_core_api_check="${{ needs.detect-changes.outputs.workplan && fromJSON(needs.detect-changes.outputs.workplan).jobs.core_api_checks || false }}"
is_pr="${{ startsWith(github.ref_name, 'pull-request/') }}"
Expand Down Expand Up @@ -679,6 +735,12 @@ jobs:
check_result "build-linux-aarch64" "$linux_expected"
check_result "build-windows" "$windows_expected"

windows_arm64_expected="skipped"
if [[ "$windows_arm64_supported" == "true" ]]; then
windows_arm64_expected="$windows_expected"
fi
check_result "build-windows-arm64" "$windows_arm64_expected"

# Sdist tests follow build selection; wheel tests follow the platform plan.
expected="skipped"
if [[ "$doc_only" != "true" && "$build_selected" == "true" ]]; then
Expand Down
1 change: 1 addition & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ repos:
- --max-retries=3
- --no-progress
files: '\.(md|rst)$'
exclude: ^qa/

# Standard hooks
- repo: https://github.com/pre-commit/pre-commit-hooks
Expand Down
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ correct.** Each package matches its own tag prefix:

| Package | Tag pattern |
| --- | --- |
| `cuda-bindings`, `cuda-python` | `v*` (e.g. `v13.3.1`) |
| `cuda-bindings`, `cuda-python` | `v*` (e.g. `v13.4.1`) |
| `cuda-core` | `cuda-core-v*` (e.g. `cuda-core-v1.1.0`) |
| `cuda-pathfinder` | `cuda-pathfinder-v*` (e.g. `cuda-pathfinder-v1.6.0`) |

Expand Down Expand Up @@ -109,7 +109,7 @@ version-check failure:
`cuda-bindings` to that same bogus version.
2. **Stale tags** (a fork that has not fetched upstream in a while): you get a
plausible-looking but wrong version, e.g. `13.0.4.dev650+g0d22cb44` when the
real latest tag is `v13.3.1`. Nothing warns you. Note there is no leading
real latest tag is `v13.4.1`. Nothing warns you. Note there is no leading
`v` — the tag prefix is stripped by `tag_regex`.
3. **No git metadata** (source zip): the build fails with
`LookupError: setuptools-scm was unable to detect version`.
Expand Down
Loading
Loading