Skip to content

feat: add SimpleStorage payload transfer abstraction - #5

Closed
Ginray wants to merge 1 commit into
mainfrom
ucx-payload
Closed

feat: add SimpleStorage payload transfer abstraction#5
Ginray wants to merge 1 commit into
mainfrom
ucx-payload

Conversation

@Ginray

@Ginray Ginray commented Aug 18, 2026

Copy link
Copy Markdown
Owner

Summary

  • add the narrow PayloadTransfer boundary for SimpleStorage large payloads
  • keep ZMQ as the default payload path and control plane
  • provide an optional UCX Tagged Host-memory implementation for RoCE RDMA
  • preserve Mooncake, YuanRong, RayStore, and existing backend ownership
  • use node-local GID/netdev and UCX capability discovery without machine-specific NIC names
  • add the developer guide for official UCX installation, TQ build/use, and RDMA lane verification
  • mark old PR-branch experiments as historical and link the current official-release result

Why

SimpleStorage needs a small data-plane boundary for large encoded payloads without changing its
ZMQ control protocol or forcing other storage backends onto the same transport. UCX is opt-in and
does not replace the storage backend or control plane.

User impact

The default remains:

payload_transfer: zmq

Users can select ucx when TQ is built with UCX support and each participating Ray node has a
matching RoCE-v2 path. The new developer guide explains the official UCX install, native build,
runtime configuration, and how to verify rc_verbs data versus TCP auxiliary traffic.

Validation

  • ruff check . passed
  • Python syntax checks passed
  • git diff --check passed
  • official, unmodified UCX v1.22.0 was used on A2-26/A2-27
  • SimpleStorage cross-node UCX PUT/GET/CLEAR passed
  • UCX logs showed tag(rc_verbs/hns_0:1) ka(tcp/enp189s0f0)
  • manager throughput sample: 128 KiB 28.75/30.75, 1 MiB 56.03/73.07,
    16 MiB 96.78/97.02 MiB/s for PUT/GET

The current native close path can leave a server-side Connection reset by remote peer warning
after the client force-closes; the data and CLEAR checks still pass.

@Ginray
Ginray force-pushed the ucx-payload branch 4 times, most recently from bf2d436 to 9207ea4 Compare August 19, 2026 02:58
@Ginray

Ginray commented Aug 19, 2026

Copy link
Copy Markdown
Owner Author

Superseded by the consolidated implementation PR #6 from ucx-payload-o1.

@Ginray Ginray closed this Aug 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant