ros2 / ros2/rclcpp

Proposal: Expose `is_data_valid` API for Loaned/Zero-Copy Messages

Open
#3,119 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
C++
Stars
805
Forks
564
Avg merge
1d 17h
Merged PRs (30d)
27

Description

Description

introduce an API at the rclcpp level that allows users to check whether loaned message memory is still valid (i.e., has not been overwritten by the middleware). this addresses a long-standing safety gap when using loaned messages and zero-copy transport with middleware implementations that reuse shared memory (e.g., Fast-DDS with data-sharing).

Motivation

when using loaned messages, especially with zero-copy pipelines over shared memory, the middleware may overwrite previously loaned memory under certain conditions:

  • the publisher and subscriber share a history cache in shared memory, so the publisher can cause changes in the subscriber's loaned memory even while the subscriber still holds a reference.
  • with PREALLOCATED_WITH_REALLOC memory policy, shared memory still in use may be reissued to new incoming samples.
  • the subscriber currently has no way to detect that its loaned data has been silently invalidated.

while loans were disabled in the executor as a short-term fix, the underlying problem remains for users who explicitly opt into loaned messages or zero-copy transports. currently, Fast-DDS provides an internal is_valid mechanism for data-sharing samples, but this is not exposed through any ROS 2 API layer.

Design / Implementation Considerations

the proposal approach is to add RMW Native Interface Pattern.

following the established pattern used by rmw_fastrtps_cpp to expose vendor-specific functionality (e.g., get_participant, get_subscriber, get_publisher), we propose a similar approach to expose data validity checks.

  • rmw_fastrtps (vendor-specific): rmw implementation that supports loaned messages with shared memory can expose a native validity check. middleware implementations that do not have this concern (e.g., Cyclone DDS, which allocates fresh memory) can either not implement this or always return true.
  • rmw interface (optional, generic): a generic function could be added to the rmw interface to allow middleware-agnostic checking. implementations that don't support or need this would return RMW_RET_UNSUPPORTED or set is_valid = true.
  • rclcpp (user-facing): expose the check through MessageInfo or the Subscription object so that users can query validity without reaching down to vendor-specific APIs like is_loaned_message_valid.
Scenario is_data_valid returns
Message was not loaned (normal copy) true (always safe)
Loaned message, memory still intact true
Loaned message, memory overwritten by publisher/pool false
Middleware does not support validity checking true (safe default)
Additional Information

several fixes have been applied or proposed (disabling loans, fixing the memory pool, returning loans via shared_ptr destructor).

  • zero-copy over shared memory inherently involves memory that the publisher can modify. this is by design in the DDS data-sharing model.
  • even with fixes to the executor and memory pool, advanced users opting into zero-copy need a mechanism to detect invalidation.
  • defense-in-depth: even if the middleware "should" not overwrite in-use memory, having a user-facing validity check provides a safety net.

in the original discussion, this check is inherently racy for true zero-copy over shared memory, memory could become invalid during a read. the is_data_valid API is therefore best suited as:

  • a post-read sanity check (read data, then verify it wasn't corrupted)
  • a debugging/diagnostic tool to identify when memory reuse issues occur
  • a guard before processing in non-real-time paths

it is not a substitute for proper loan lifetime management (returning loans via shared_ptr destructor, etc.).

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the rclcpp MessageInfo and Subscription API areas, then compare the rmw_fastrtps_cpp native interface pattern mentioned in the proposal. Determine the intended boundary between vendor-specific, generic rmw, and user-facing rclcpp checks; done means the API behavior is defined for copied, valid loaned, invalidated loaned, and unsupported middleware cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
api, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.