Skip to content

feat: add SimpleStorage payload transfer abstraction - #6

Open
Ginray wants to merge 2 commits into
mainfrom
ucx-payload-o1
Open

feat: add SimpleStorage payload transfer abstraction#6
Ginray wants to merge 2 commits into
mainfrom
ucx-payload-o1

Conversation

@Ginray

@Ginray Ginray commented Aug 19, 2026

Copy link
Copy Markdown
Owner

Summary

  • add the narrow PayloadTransfer boundary for SimpleStorage large payloads
  • keep ZMQ as the default payload path and control plane
  • provide an optional UCX Tagged Host-memory implementation for RoCE RDMA
  • preserve Mooncake, YuanRong, RayStore, and existing backend ownership
  • discover node-local RDMA/GID/netdev capabilities without machine-specific NIC names
  • keep the MR focused on implementation and necessary automated tests

Why

SimpleStorage needs a small data-plane boundary for large encoded payloads without changing its
ZMQ control protocol or forcing other storage backends onto the same transport. UCX is opt-in and
does not replace the storage backend or control plane.

User impact

The default remains:

payload_transfer: zmq

Users can select ucx when TQ is built with UCX support and each participating Ray node has a
matching RoCE-v2 path. UCX uses RC for the payload data lane; TCP may be used only as the UCX
wireup/auxiliary lane when the environment requires it.

Validation

  • ruff check . passed
  • git diff --check passed
  • protocol and lifecycle tests are retained in tests/test_payload_transfer.py
  • official, unmodified UCX v1.22.0 was validated separately on A2-26/A2-27
  • SimpleStorage cross-node UCX PUT/GET/CLEAR passed in the recorded validation

The current native close path can leave a server-side Connection reset by remote peer warning
after a client force-closes; the data and CLEAR checks still pass.

@Ginray
Ginray force-pushed the ucx-payload-o1 branch 3 times, most recently from 1515b65 to 3e646fd Compare August 19, 2026 09:49
@Ginray
Ginray force-pushed the ucx-payload-o1 branch 5 times, most recently from 31047db to 429baf4 Compare August 27, 2026 13:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant