Skip to content

feat(cuda.core): add cluster scheduling policy to LaunchConfig - #2703

Open
atiaomar1978-hub wants to merge 16 commits into
NVIDIA:mainfrom
atiaomar1978-hub:feat/launch-config-cluster-scheduling-policy-2629-signed
Open

feat(cuda.core): add cluster scheduling policy to LaunchConfig#2703
atiaomar1978-hub wants to merge 16 commits into
NVIDIA:mainfrom
atiaomar1978-hub:feat/launch-config-cluster-scheduling-policy-2629-signed

Conversation

@atiaomar1978-hub

Copy link
Copy Markdown
Contributor

Description

closes #2629

Add cluster_scheduling_policy_preference to LaunchConfig, mapping to
CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE via
ClusterSchedulingPolicyType (DEFAULT, SPREAD, LOAD_BALANCING).

Unlike stream synchronization policy (#2628), cluster scheduling policy is
documented as valid for graph nodes and kernel launches, so LaunchConfig is
the correct surface.

This is a signed re-issue of #2702 (same change, linear history on current
main). Commits are SSH-signed. GitHub would not reopen #2668.

Checklist

  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

Test plan

  • Native attribute mapping for DEFAULT / SPREAD / LOAD_BALANCING
  • Getter/setter round-trip for each policy
  • Driver enum acceptance; invalid value rejection; CC < 9.0 gate
  • Combined cluster + policy attributes
  • Hopper+ launch() smoke for all three policies
  • RunPod H100 80GB HBM3 / driver 580 / CTK 13.3 — 13 passed, 0 skipped
  • pre-commit.ci stubgen-pyx stubs regenerated

@copy-pr-bot

copy-pr-bot Bot commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions Bot added the cuda.core Everything related to the cuda.core module label Aug 26, 2026
atiaomar1978-hub and others added 4 commits August 25, 2026 17:19
Expose CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE on
LaunchConfig via ClusterSchedulingPolicyType, with validation, CC >= 9.0
gating, and tests mapping to the native launch attribute.

Closes NVIDIA#2629

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Omar Atie <atiaomar1978@gmail.com>
Assert LaunchConfig emits both CU_LAUNCH_ATTRIBUTE_CLUSTER_DIMENSION
and CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE when set.

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Omar Atie <atiaomar1978@gmail.com>
Use init_cuda with an inline noop kernel so test_launcher.py does not
depend on the get_saxpy_kernel_cubin fixture from test_module.py.

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Omar Atie <atiaomar1978@gmail.com>
…rough

Align NVIDIA#2629 tests with reviewer guidance: round-trip each policy on
LaunchConfig and exercise launch() for DEFAULT/SPREAD/LOAD_BALANCING.

Co-authored-by: Cursor <cursoragent@cursor.com>
Signed-off-by: Omar Atie <atiaomar1978@gmail.com>
@atiaomar1978-hub

atiaomar1978-hub commented Aug 26, 2026

Copy link
Copy Markdown
Contributor Author

Hi @lijinf2 — when you have a moment, could you please review this PR?

This is the signed re-issue of #2702 (GitHub would not reopen #2668). It implements the plan we aligned on in #2629: LaunchConfig.cluster_scheduling_policy_preference maps to CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE (DEFAULT / SPREAD / LOAD_BALANCING), with CC >= 9.0 gating.

Tests follow your guidance (getter/setter plus a launch() run-through; no attempt to observe the live scheduler). On RunPod H100 80GB HBM3 (CC 9.0, driver 580, CTK 13.3) the cluster-policy suite is 13 passed, 0 skipped.

I do not have permission to set assignee or milestone on this account; cuda.core 1.3.0 would match the related work if that is still the right milestone.

Sorry for the force-push on this branch. I know rewriting published history is against the project rules. I did it only so the commits would be SSH-signed and show as Verified on GitHub. I will not force-push again.

@lijinf2

lijinf2 commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

/ok to test c6c283b

@lijinf2 lijinf2 added the feature New feature or request label Aug 27, 2026
@lijinf2 lijinf2 added this to the cuda.core next milestone Aug 27, 2026
@github-actions

Copy link
Copy Markdown

CI test_subpackage_exports_are_documented[typing] failed because the
new typing export was missing from docs/source/*.rst.

Signed-off-by: Omar Atie <atiaomar1978@gmail.com>
@atiaomar1978-hub

Copy link
Copy Markdown
Contributor Author

Posted a docs fix for the CI failure on test_subpackage_exports_are_documented[typing].

ClusterSchedulingPolicyType was exported from cuda.core.typing but missing from docs/source/*.rst. Commit 39c9fc0843 adds it to api_private.rst (SSH-signed / Verified).

RunPod (RTX 2000 Ada, CC 8.9 — L4 community was out of stock; same compute capability as the failing L4 jobs):

  • test_subpackage_exports_are_documented[typing] passed
  • cluster launcher suite: 10 passed, 3 skipped (launch smokes require CC >= 9.0)

PR metadata is now set (cuda.core, feature, assignee, milestone).

@atiaomar1978-hub

atiaomar1978-hub commented Aug 27, 2026

Copy link
Copy Markdown
Contributor Author

Hi @lijinf2 — could you please trigger CI again on 39c9fc0843?

The previous /ok to test covered c6c283bb5d3f. That matrix failed; this SHA has the docs fix and has not gotten a new GPU run yet.

Why CI failed

Every GPU job died on the same assertion — not on LaunchConfig or Hopper launch:

tests/test_api_docs_consistency.py::test_subpackage_exports_are_documented[typing] FAILED
E   AssertionError: public by cuda.core.typing.__all__ but missing from docs/source/*.rst:
E       ['ClusterSchedulingPolicyType']

ClusterSchedulingPolicyType was added to cuda.core.typing.__all__ but was not listed in cuda_core/docs/source/api_private.rst (the autosummary the consistency test reads). Example: linux-64 / py3.10 / CUDA 13.3 wheels / L4 — 1 failed, launcher cluster tests passed or skipped by CC.

Why it works again

39c9fc0843 adds the missing autosummary line:

typing.ClusterSchedulingPolicyType

That name is now present in docs/source/*.rst, so the export-vs-docs check passes. Runtime tests were already green.

RunPod replay (pod 7xp9vqqo44kk1s, RTX 2000 Ada, CC 8.9)

L4 community was out of stock; Ada is the same CC 8.9 as the failing L4 jobs. HEAD on the pod was 39c9fc0.

############ CI FAILURE: docs export consistency (typing) ############
tests/test_api_docs_consistency.py::test_subpackage_exports_are_documented[typing] PASSED [100%]
======================= 1 passed, 5 deselected in 1.10s ========================

############ CLUSTER SCHEDULING POLICY TESTS (CI-like GPU) ############
compute_capability ComputeCapability(major=8, minor=9)
tests/test_launcher.py::test_to_native_launch_config_cluster_scheduling_policy[0] PASSED
tests/test_launcher.py::test_to_native_launch_config_cluster_scheduling_policy[1] PASSED
tests/test_launcher.py::test_to_native_launch_config_cluster_scheduling_policy[2] PASSED
tests/test_launcher.py::test_to_native_launch_config_cluster_scheduling_policy_accepts_driver_enum PASSED
tests/test_launcher.py::test_launch_config_cluster_scheduling_policy_invalid PASSED
tests/test_launcher.py::test_launch_config_cluster_scheduling_policy_rejects_pre_hopper_cc PASSED
tests/test_launcher.py::test_to_native_launch_config_cluster_and_policy PASSED
tests/test_launcher.py::test_launch_config_cluster_scheduling_policy_getter_setter[0] PASSED
tests/test_launcher.py::test_launch_config_cluster_scheduling_policy_getter_setter[1] PASSED
tests/test_launcher.py::test_launch_config_cluster_scheduling_policy_getter_setter[2] PASSED
tests/test_launcher.py::test_launch_cluster_scheduling_policy_smoke[0] SKIPPED
tests/test_launcher.py::test_launch_cluster_scheduling_policy_smoke[1] SKIPPED
tests/test_launcher.py::test_launch_cluster_scheduling_policy_smoke[2] SKIPPED
=========== 10 passed, 3 skipped, 61 deselected, 1 warning in 0.74s ============

Launch smokes skip on CC < 9.0, same as the L4 CI jobs. Pod has been deleted.

Full pod log (nvidia-smi + pytest; apt/pip noise omitted)
Thu Aug 27 21:56:33 2026
NVIDIA-SMI 570.195.03   Driver Version: 570.195.03   CUDA Version: 12.8
GPU: NVIDIA RTX 2000 Ada Generation

HEAD: 39c9fc0 docs(cuda.core): document ClusterSchedulingPolicyType in API RST
Fetching mini CUDA toolkit 13.3.0...
#define CUDA_VERSION 13030
cuda.core 0.0.0

tests/test_api_docs_consistency.py::test_subpackage_exports_are_documented[typing] PASSED
======================= 1 passed, 5 deselected in 1.10s ========================

compute_capability ComputeCapability(major=8, minor=9)
10 passed, 3 skipped, 61 deselected, 1 warning in 0.74s

Thank you.

@lijinf2

lijinf2 commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

/ok to test 39c9fc0

@lijinf2

lijinf2 commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

@atiaomar1978-hub Hey, there are 7 test cases, and it looks like some are redundant. Could we remove redundancy and merge them into at most 3 cases to reduce code maintenance overhead going forward?

@atiaomar1978-hub

Copy link
Copy Markdown
Contributor Author

I will do now @lijinf2

Merge seven overlapping LaunchConfig cluster-scheduling tests into
mapping, rejection, and Hopper launch smoke to cut maintenance.

Signed-off-by: Omar Atie <atiaomar1978@gmail.com>
@atiaomar1978-hub

Copy link
Copy Markdown
Contributor Author

@lijinf2 folded the seven cluster-policy tests into three in 1a78ad5cd1 (Verified):

  1. test_launch_config_cluster_scheduling_policy — ctor / getter-setter / driver enum / native attrs for DEFAULT, SPREAD, LOAD_BALANCING (including cluster + policy together)
  2. test_launch_config_cluster_scheduling_policy_rejected — invalid value and CC < 9.0
  3. test_launch_cluster_scheduling_policy_smokelaunch() for all three policies on Hopper+

Could you /ok to test 1a78ad5cd1cf0294d91994b0214ec71bdf9bdd2f when you have a moment?

@lijinf2

lijinf2 commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

/ok to test 1a78ad5

CUDA 13.0.2's CUlaunchAttributeValue setter reads .value, so a raw int
breaks native LaunchConfig conversion in CI.

Signed-off-by: Omar Atie <atiaomar1978@gmail.com>
@atiaomar1978-hub

Copy link
Copy Markdown
Contributor Author

@lijinf2 CUDA 13.0.2 jobs failed on test_launch_config_cluster_scheduling_policy (AttributeError — that setter reads .value, so a raw int breaks). 3a90298eb1 passes CUclusterSchedulingPolicy instead.

RunPod replay (RTX 2000 Ada, bindings 13.0.3): mapping + reject PASSED; smoke SKIPPED on CC 8.9 (already passed on CI H100).

Could you please /ok to test 3a90298eb1e788047e702076b81dcdb28073e699?

…-2629-signed

Signed-off-by: Omar Atie <atiaomar1978@gmail.com>
@lijinf2

lijinf2 commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

/ok to test 3a90298

@copy-pr-bot

copy-pr-bot Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

/ok to test 3a90298

@lijinf2, there was an error processing your request: E2

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/

Comment thread cuda_core/cuda/core/_launch_config.pxd Outdated
Comment thread cuda_core/cuda/core/_launch_config.pyi Outdated
Comment thread cuda_core/cuda/core/_launch_config.pyi Outdated
Comment thread cuda_core/cuda/core/_launch_config.pyi Outdated
@lijinf2

lijinf2 commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

/ok to test c15bf56

@copy-pr-bot

copy-pr-bot Bot commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

/ok to test c15bf56

@lijinf2, there was an error processing your request: E2

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/2/

@lijinf2

lijinf2 commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

/ok to test d185a86

@atiaomar1978-hub

Copy link
Copy Markdown
Contributor Author

Thanks @lijinf2 for triggering the CI builds and tests 👍

Comment thread cuda_core/cuda/core/_launch_config.pyx Outdated
'cluster_scheduling_policy_preference',
)

_CLUSTER_SCHED_POLICY_NAMES = (

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this variable and reuse _CLUSTER_SCHED_POLICY_TO_DRIVER dictionary keys. In future maintainance we hope less variables.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in e85fafe — removed _CLUSTER_SCHED_POLICY_NAMES; validation now uses _CLUSTER_SCHED_POLICY_TO_DRIVER.keys().

Comment thread cuda_core/cuda/core/_launch_config.pyx Outdated
)


def _cluster_sched_policy_to_driver(value):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete this function to use _CLUSTER_SCHED_POLICY_TO_DRIVER[value] directly.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in e85fafe — removed _cluster_sched_policy_to_driver; lookup is _CLUSTER_SCHED_POLICY_TO_DRIVER[value] via LaunchConfig._cluster_sched_policy_driver_value().

Comment thread cuda_core/cuda/core/_launch_config.pyx Outdated
"LOAD_BALANCING",
)

_CLUSTER_SCHED_POLICY_TO_DRIVER = {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's combine _CLUSTER_SCHED_POLICY_TO_DRIVER, and _validate_cluster_scheduling_policy_preference into a unified function (maybe _validate_cluster_scheduling_policy_preference or better name)?

Also, please put the unified function under LaunchConfig as a member function not global.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in e85fafe — merged validation into LaunchConfig._validate_cluster_scheduling_policy_preference (member method).

Comment thread cuda_core/cuda/core/_launch_config.pyx Outdated
"cluster launch attributes are not supported on devices with "
f"compute capability < 9.0 (got {cc})"
)
if cluster is not None:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's keep lines 142-150 unchanged and add compute capability checking to the unified function.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in e85fafe — lines 142–150 unchanged; CC check for policy is also in the member validator.

attr.value.clusterSchedulingPolicyPreference = _cluster_sched_policy_to_driver(
config.cluster_scheduling_policy_preference
)
attrs.append(attr)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we set twice? one at line 217-223, and the other at line 285-292?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The attribute is appended in two conversion paths until all modules are cythonized: the cdef _to_native_launch_config (cydriver structs) and the cpdef module helper (Python driver.CUlaunchAttribute objects). e85fafe factors shared driver lookup into LaunchConfig._cluster_sched_policy_driver_value().

Comment thread cuda_core/tests/test_launcher.py Outdated
assert (attr.value.clusterDim.x, attr.value.clusterDim.y, attr.value.clusterDim.z) == (2, 2, 2)


_CLUSTER_SCHED_POLICIES = (

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove _CLUSTER_SCHED_POLICIES and import.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in e85fafe — tests iterate _CLUSTER_SCHED_POLICY_TO_DRIVER keys instead of a local tuple.

@lijinf2

lijinf2 commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

atiaomar1978-hub

@atiaomar1978-hub Hey, are you a human contributor or a code agent contributor? Your replies and code style read like automatic agent-generated content, with little human review/refinement.

@atiaomar1978-hub

Copy link
Copy Markdown
Contributor Author

human + agent together and integration with RunPod to Test @lijinf2

…onfig

Move validation and driver lookup onto LaunchConfig members, drop duplicate
policy name constants, and reuse _CLUSTER_SCHED_POLICY_TO_DRIVER in tests.

Signed-off-by: Omar Atie <atiaomar1978@gmail.com>
@atiaomar1978-hub

Copy link
Copy Markdown
Contributor Author

Addressed latest review feedback in e85fafe:

  • Moved cluster policy validation onto LaunchConfig._validate_cluster_scheduling_policy_preference (uses _CLUSTER_SCHED_POLICY_TO_DRIVER keys; CC check included).
  • Removed _CLUSTER_SCHED_POLICY_NAMES, _cluster_sched_policy_to_driver, and test-local _CLUSTER_SCHED_POLICIES.
  • Shared driver lookup via LaunchConfig._cluster_sched_policy_driver_value() for both native conversion paths.

RunPod (H100, CC 9.0, cuda.bindings 13.3.1): 3/3 cluster scheduling policy tests passed.

Ready for /ok to test e85fafe when convenient.

@lijinf2

lijinf2 commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

/ok to test e85fafe

@atiaomar1978-hub

Copy link
Copy Markdown
Contributor Author

The failing job Test win-64 / Python 3.14, CUDA 13.3.0 (wheels), GPU h100 (x2) (MCDM) (run) does not look related to this PR's code changes.

It failed during Configure driver mode (ci/tools/configure_driver_mode.ps1) while switching the Windows runner to MCDM and cycling the dual H100 devices. The job never reached artifact download or Run cuda.core tests.

On the same commit (e85fafe), builds, pre-commit, API checks, and the other Windows/Linux GPU test jobs passed. This matrix entry is the only dual-H100 + MCDM Windows job in the PR matrix, so this looks like a runner/infra flake rather than a regression from the cluster scheduling policy changes.

Happy to re-run that job if helpful. Cluster policy coverage was verified separately on RunPod H100 (3/3 tests passed).

@atiaomar1978-hub

Copy link
Copy Markdown
Contributor Author

@lijinf2 When you have a moment, could you please take another look at the latest review feedback addressed in e85fafe?

The only failing check on that commit was the Windows dual-H100 (MCDM) job, which failed during Configure driver mode before any cuda.core tests ran. That looks like a runner/infra issue rather than a regression from this PR. If you agree, a re-run of that job (or a fresh /ok to test) would be appreciated.

Cluster scheduling policy coverage was verified on RunPod H100 (3/3 tests passed). Thanks again for the detailed review.

@atiaomar1978-hub

Copy link
Copy Markdown
Contributor Author

@lijinf2 Friendly follow-up on e85fafe.

All review feedback from the last round has been addressed, and CI is now green on that commit — including the Windows dual-H100 (MCDM) job, which passed on re-run (previously failed during Configure driver mode before tests started).

Summary of e85fafe:

  • Validation consolidated on LaunchConfig._validate_cluster_scheduling_policy_preference
  • Removed duplicate policy constants/helpers; tests use _CLUSTER_SCHED_POLICY_TO_DRIVER keys
  • Shared driver lookup via LaunchConfig._cluster_sched_policy_driver_value()
  • RunPod H100: 3/3 cluster scheduling policy tests passed

When you have time, could you please take another look and let me know if anything else is needed before merge? Thank you.

Comment thread cuda_core/cuda/core/_launch_config.pyi Outdated
Whether to allow programmatic stream serialization / PDL (default: False)
cluster_scheduling_policy_preference : str, optional
Cluster scheduling policy for the launch: ``"DEFAULT"``,
``"SPREAD"``, or ``"LOAD_BALANCING"`` (default: None)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix doc to make sure None case is specific (not used or DEFAULT?)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, why do we allow 'None' here? Is it different from "DEFAULT" option?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in f80f638 — docstrings now spell this out:

  • None (default): omit CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE; the driver uses the kernel function's default policy.
  • "DEFAULT": emit the launch attribute with CU_CLUSTER_SCHEDULING_POLICY_DEFAULT.

They are intentionally different: omission leaves the attribute unset; "DEFAULT" sets it explicitly (useful when overriding a kernel-level non-default policy at launch time).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes — None and "DEFAULT" differ:

  • None: no launch attribute is sent; the kernel function's default applies.
  • "DEFAULT": the launch attribute is sent with CU_CLUSTER_SCHEDULING_POLICY_DEFAULT.

Both are documented in f80f638.

Comment thread cuda_core/cuda/core/_launch_config.pyx Outdated
"DEFAULT": driver.CUclusterSchedulingPolicy.CU_CLUSTER_SCHEDULING_POLICY_DEFAULT,
"SPREAD": driver.CUclusterSchedulingPolicy.CU_CLUSTER_SCHEDULING_POLICY_SPREAD,
"LOAD_BALANCING": driver.CUclusterSchedulingPolicy.CU_CLUSTER_SCHEDULING_POLICY_LOAD_BALANCING,
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's move this to be a static member variable under class LaunchConfig

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in f80f638_CLUSTER_SCHED_POLICY_TO_DRIVER is now a class attribute on LaunchConfig; validation and driver lookup use LaunchConfig._CLUSTER_SCHED_POLICY_TO_DRIVER.

Comment thread cuda_core/cuda/core/_launch_config.pyx Outdated
if config.cluster_scheduling_policy_preference is not None:
attr = driver.CUlaunchAttribute()
attr.id = driver.CUlaunchAttributeID.CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE
# 13.0.2 setter reads .value; pass FastEnum, not a raw int.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove the comment. Code already explains itself.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in f80f638 — removed that comment from the cpdef conversion path.

@lijinf2

lijinf2 commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

@atiaomar1978-hub I just sent out the suggested edits/comments. Other than that, the PR looks in good shape. Thanks!

Move _CLUSTER_SCHED_POLICY_TO_DRIVER onto LaunchConfig, clarify None
versus "DEFAULT" in docs, restore cluster-only CC check in __init__,
and drop the redundant cpdef conversion comment.

Signed-off-by: Omar Atie <atiaomar1978@gmail.com>
…-2629-signed

Integrate upstream LaunchConfig priority support (NVIDIA#2706) with cluster
scheduling policy while preserving PR commit history.

Signed-off-by: Omar Atie <atiaomar1978@gmail.com>
@atiaomar1978-hub

atiaomar1978-hub commented Sep 9, 2026

Copy link
Copy Markdown
Contributor Author

@lijinf2 — addressed the latest review on f80f638991 (no force-push; merge commit preserves history):

  1. _CLUSTER_SCHED_POLICY_TO_DRIVER — moved onto LaunchConfig as a class attribute.
  2. Docstring — clarified that None omits the launch attribute (driver/kernel default), while "DEFAULT" sets the driver default explicitly via the attribute.
  3. Comment removed — dropped the redundant cpdef conversion note.
  4. Merged main — integrated upstream #2706 (LaunchConfig.priority) alongside cluster scheduling policy; resolved conflicts in _launch_config.* and test_object_protocols.py.

RunPod (H100 80GB HBM3, CC 9.0, driver 580, CTK 13.3 / bindings 13.3.1):

3 passed — test_launch_config_cluster_scheduling_policy
         test_launch_config_cluster_scheduling_policy_rejected
         test_launch_cluster_scheduling_policy_smoke

Could you please re-review and trigger CI with /ok to test f80f638991?

@lijinf2 lijinf2 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. thanks!

@lijinf2

lijinf2 commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

/ok to test f80f638

@lijinf2
lijinf2 enabled auto-merge (squash) September 9, 2026 21:37
Align _launch_config.pyi with stubgen-pyx 0.2.22 after moving the
cluster policy map onto LaunchConfig.

Signed-off-by: Omar Atie <atiaomar1978@gmail.com>
auto-merge was automatically disabled September 9, 2026 22:09

Head branch was pushed to by a user without write access

@atiaomar1978-hub

Copy link
Copy Markdown
Contributor Author

/ok to test f80f638

@lijinf2 could you re-run the CI build ?

@lijinf2

lijinf2 commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

/ok to test a44088c

@lijinf2

lijinf2 commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

@atiaomar1978-hub pre-commit check is still failing. CI needs that passed to let pr merge.

Use the repo-relative stubgen source path so pre-commit stubgen-pyx
does not rewrite _launch_config.pyi on every platform.

Signed-off-by: Omar Atie <atiaomar1978@gmail.com>
@atiaomar1978-hub

Copy link
Copy Markdown
Contributor Author

@lijinf2 pre-commit is green now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cuda.core Everything related to the cuda.core module feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEA]: Support CU_LAUNCH_ATTRIBUTE_CLUSTER_SCHEDULING_POLICY_PREFERENCE in LaunchConfig

2 participants