ros2 / ros2/system_tests

:man_farmer: `test_security` -> `test_processes_finished_gracefully` and `test_subscriber_terminates_in_a_finite_amount_of_time` tests failing on nightlies `linux` + `linux-rhel` and windows repeated

Open
#498 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
45
Forks
57
Avg merge
9h 19m
Merged PRs (30d)
2

Description

Bug report

Required Info:

  • Operating System:
    • jammy + Linux-Rhel + Windows flaky
  • Installation type:
    • Source, buildfarm
  • Version or commit hash:
    • master
  • DDS implementation:
    • Default rmw, Fast-RTPS
Steps to reproduce issue

Run a buildfarm job on linux rhel or on ubuntu jammy.

Expected behavior

All test pass.

Actual behavior

Some security tests fail consistently:

test_security.TestSecurePublisherSubscriberAfterShutdown.test_processes_finished_gracefully
test_security.TestSecurePublisherSubscriber.test_subscriber_terminates_in_a_finite_amount_of_time

Additional information

This started failing consistently on the ubuntu jammy jobs after the transition from python3.9 to 3.10.
https://ci.ros2.org/view/nightly/job/nightly_linux_debug/2246/

This a link to the failing section of the test:
https://github.com/ros2/system_tests/blob/4fe90b6d72ee33fb0a1a22dec7cf393bafa537af/test_security/test/test_secure_publisher_subscriber.py.in#L76
and
https://github.com/ros2/system_tests/blob/4fe90b6d72ee33fb0a1a22dec7cf393bafa537af/test_security/test/test_secure_publisher_subscriber.py.in#L84

Possible clues

It has been failing constantly since 2 months ago in rhel.
First getting some clues from RHEL. It started showing there 2 months ago between the transition of these two jobs:
https://ci.ros2.org/view/nightly/job/nightly_linux-rhel_debug/1012/
https://ci.ros2.org/view/nightly/job/nightly_linux-rhel_debug/1013/

And the changes between one job and the other are these PRs:
https://github.com/ros2/rosidl_python/pull/149
https://github.com/ros2/geometry2/pull/496
https://github.com/eProsima/Fast-DDS/pull/2399
https://github.com/ros2/rclcpp/pull/1862
https://github.com/ament/ament_index/pull/83

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with test_security/test/test_secure_publisher_subscriber.py.in at the referenced lines 76 and 84, then run the two failing tests in a buildfarm job on Ubuntu Jammy, RHEL, and Windows. Compare the failing nightly jobs and the listed dependency changes to identify the regression. Done means both tests pass reliably across the affected platforms.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
distributed-systems, security, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.