ROS2 Daemon fails to respond

Bug report

Summary

Running ros2 topic list once results in all subsequent ros2 * commands to fail with the following error:

Traceback (most recent call last):
  File "/opt/ros/foxy/bin/ros2", line 11, in <module>
    load_entry_point('ros2cli==0.9.11', 'console_scripts', 'ros2')()
  File "/opt/ros/foxy/lib/python3.8/site-packages/ros2cli/cli.py", line 67, in main
    rc = extension.main(parser=parser, args=args)
  File "/opt/ros/foxy/lib/python3.8/site-packages/ros2topic/command/topic.py", line 41, in main
    return extension.main(args=args)
  File "/opt/ros/foxy/lib/python3.8/site-packages/ros2topic/verb/list.py", line 38, in main
    with NodeStrategy(args) as node:
  File "/opt/ros/foxy/lib/python3.8/site-packages/ros2cli/node/strategy.py", line 52, in __enter__
    self._daemon_node.__enter__()
  File "/opt/ros/foxy/lib/python3.8/site-packages/ros2cli/node/daemon.py", line 116, in __enter__
    methods = self._proxy.system.listMethods()
  File "/usr/lib/python3.8/xmlrpc/client.py", line 1109, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib/python3.8/xmlrpc/client.py", line 1450, in __request
    response = self.__transport.request(
  File "/usr/lib/python3.8/xmlrpc/client.py", line 1153, in request
    return self.single_request(host, handler, request_body, verbose)
  File "/usr/lib/python3.8/xmlrpc/client.py", line 1166, in single_request
    resp = http_conn.getresponse()
  File "/usr/lib/python3.8/http/client.py", line 1348, in getresponse
    response.begin()
  File "/usr/lib/python3.8/http/client.py", line 316, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.8/http/client.py", line 285, in _read_status
    raise RemoteDisconnected("Remote end closed connection without"
http.client.RemoteDisconnected: Remote end closed connection without response

Running ros2 daemon stop fails with the same error; the only way to stop the daemon is to run killall _ros2_daemon.

I am able to run ros2 topic echo .... and ros2 topic pub ... successfully, as long as I run them before running ros2 topic list. As soon as I run ros2 topic list once, all subsequent commands fail with the error above.

Required Info:

• Operating System:
  • Ubuntu 20.04
• Installation type:
  • Binary
• Version or commit hash:
  • 0.9.11
• DDS implementation:
  • rmw_cyclonedds_cpp
  • rmw_fastrtps_cpp
  • (All computers on the network use the same one, but I've tried using both)

Steps to reproduce issue

I have 2 computers installed in my robot, both running Ubuntu 20.04. One publishes a ROS2 API that I need to expose to ROS Noetic on the other. For nomenclature purposes:

• "Main PC" refers to the one connected to the robot's MCU. It has ROS Noetic + Foxy installed on it, and runs the ROS1 master
• "Backpack PC" refers to the secondary computer, which publishes the ROS2 API I need to expose

The main PC has the ros1_bridge package installed on it, though at this point I have not even been able to use it.

Both PCs are configured to use CycloneDDS and have the following envars set:

export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
export CYCLONEDDS_URI=file:///path/to/cyclone_dds.xml

cyclone_dds.xml contents:

<?xml version="1.0" encoding="UTF-8" ?>
<CycloneDDS xmlns="https://cdds.io/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="https://cdds.io/config https://raw.githubusercontent.com/eclipse-cyclonedds/cyclonedds/master/etc/cyclonedds.xsd">
  <Domain id="any">
    <General>
      <NetworkInterfaceAddress>eno1</NetworkInterfaceAddress>
      <AllowMulticast>true</AllowMulticast>
    </General>
    <Internal>
      <LeaseDuration>5 min</LeaseDuration>
    </Internal>
  </Domain>
</CycloneDDS>

Both computers have eno1 statically configured to unique IP addresses on the same subnet. I've tried 2 different subnets in case there was a conflict with something else on my network, but there was no effect.

Expected behavior

Running ros2 topic list multiple times should work, running ros2 topic echo ... after running ros2 topic list should echo the specified topic, ros2 daemon stop should be able to stop the daemon.

Actual behavior

As soon as I run ros2 topic list once all subsequent commands fail with http.client.RemoteDisconnected: Remote end closed connection without response. Killing the ROS2 daemon with killall _ros2_daemon temporarily fixes the problem, until I run ros2 topic list again.

If you need any additional information, or want me to run additional tests on one/both computers please let me know and I'll provide whatever information I can.

Thanks a lot for the help! I'm hopeful that this is just something simple I've misconfigured and there's an easy fix.

Start with ros2cli/node/daemon.py and ros2cli/node/strategy.py, then trace the ros2topic/verb/list.py path shown in the traceback. Reproduce the failure by running ros2 topic list followed by another ros2 command or ros2 daemon stop. Done means repeated CLI commands work and the daemon can be stopped without killing _ros2_daemon.