ROCm / ROCm/hipObject

x-amz-rdma-token: example appends :address:size to the token; document the normative header format

Open
#39 1 comment 2 reactions 2 assignees View on GitHub

@sbates130272 is already working on this.

Since Sep 18, 2026.

  • #41 by @copilot-swe-agent — open
Dominant language
C++
Stars
9
Forks
1
Avg merge
1d 9h
Merged PRs (30d)
7

Description

Summary

The reference S3 callback in examples/s3_curl_ops.cpp does not send the token that encodeRdmaToken() produces. It appends two extra fields to the header value:

std::snprintf(rdmaHeader, sizeof(rdmaHeader), "%.*s:%016lx:%016lx",
              static_cast<int>(tokenLen), token,
              reinterpret_cast<uintptr_t>(cfg->devPtr),
              static_cast<unsigned long>(cfg->objectSize));

so the header on the wire is <token-hex>:<address>:<size> rather than <token-hex>.

Why this looks wrong

Both appended values are already carried inside the token. RdmaToken (src/rdma/token.h) holds remoteAddr and length, and encodeRdmaToken() (src/rdma/token.cpp) serialises them into the fixed-width payload:

constexpr size_t kTokenBinaryLen = 1 + 4 + 16 + 4 + 8 + 8 + 1 + 2;  // 44 bytes
constexpr size_t kTokenHexLen = kTokenBinaryLen * 2;                // 88 hex chars

The suffix therefore duplicates in-token fields, and it turns a fixed-length header value into a variable-length one. A peer that validates the token as a fixed-length hex string — which kTokenHexLen suggests is the intent, and which decodeRdmaTokenHex() enforces on the receiving side — will reject the suffixed form outright. That makes the reference example the one thing an integrator cannot copy verbatim.

Requests
  1. Emit the token as-is in the example. ops.sendRequest() already receives token and tokenLen from the library; the example should place exactly those bytes in x-amz-rdma-token. The token being self-describing is what makes the callback contract implementable — an application should never need to know that a buffer address or a length has to be stapled on afterwards.

  2. Write down the normative header format. docs/interop.rst currently says only that applications "embed a hex-encoded RDMA token in x-amz-rdma-token", which does not settle the question. Please document, in one authoritative place:

    • field order, width and endianness of each token field
    • the transport byte values (0x00 DC, 0x01 RC) and any other enumerated values
    • the total encoded length, and that it is fixed
    • explicitly, whether any suffix or additional :-separated component is ever permitted in the header value, and if so what parses it
  3. While that section is being written, the accepted x-amz-rdma-reply forms would be worth pinning down the same way. docs/interop.rst mentions legacy ok/err tags, numeric HTTP codes, and 200:<server-token-hex>; an implementer needs to know which of those a compliant client must accept and which are deprecated.

Happy to review a draft of the format section if that is useful.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.