ros2 / ros2/system_tests

:farmer: `test_cli_remapping` flaky test

Open
#511 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
45
Forks
57
Avg merge
9h 19m
Merged PRs (30d)
2

Description

Bug report

Required Info:

  • Operating System:
    • Windows (and sometimes Ubuntu 20.04)
  • Installation type:
    • Source
  • Version or commit hash:
    • Rolling / Foxy
Steps to reproduce issue
  • Run a build in nightly windows or nightly repeated jobs
  • See test_cli_remapping fail if you're lucky
Expected behavior
  • test_cli_remapping should pass
Actual behavior
  • test_cli_remapping is failing sometimes
Additional information

Log output:
Although most of the tests have different names, the log output is very similar in all of them:

Traceback (most recent call last):
  File "C:\ci\ws\src\ros2\system_tests\test_cli_remapping\test\test_cli_remapping.py", line 149, in test_topic_and_service_replacement
    self.assertIn(topic_and_service_replacement, self.get_topics())
AssertionError: '/remapped/ts5907_00_20_08' not found in ['/fully/qualified/name', '/ns/s5749_00_20_08/original_node_name/private/name', '/ns/s5749_00_20_08/relative/name', '/original/namespace/node_6563_00_20_08/private/name', '/original/namespace/original_node_name/private/name', '/original/namespace/relative/name', '/parameter_events', '/remapped/t1668_00_20_08', '/rosout']

Reference builds:

Buildfarmer stats

Flakiness ratio in the last 30 days:

  • Job: Fci__nightly-connext_ubuntu_focal_amd64
    • Last 30 builds: 3 builds failed, 10% with error
    • Last 7 builds: 1 builds failed, 14.28% with error
  • Job: nightly_win_rel
    • Last 30 builds: 5 builds failed, 16.66% with error
    • Last 7 builds: 1 builds failed, 14.28% with error
  • Job: nightly_win_rep
    • Last 30 builds: 10 builds failed, 33.33% with error
    • Last 7 builds: 3 builds failed, 42.85% with error

This test has failed on the following jobs:

Job Last time it happened First time it happened
nightly_win_rep 2023-01-16 2022-01-26
Fci__nightly-connext_ubuntu_focal_amd64 2023-01-14 2021-06-05
nightly_linux-rhel_repeated 2022-12-14 2022-02-05
nightly_linux-aarch64_repeated 2022-11-12 2022-02-01
nightly_linux_repeated 2022-11-12 2022-02-13
nightly_win_deb 2022-10-15 2022-10-15
nightly_win_rel 2022-07-10 2022-03-08

They're mostly Repeated and windows jobs

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with system_tests/test_cli_remapping/test/test_cli_remapping.py, especially the failure at line 149, and reproduce the test in the referenced repeated Windows or Ubuntu jobs. Compare the logged topic and service names across failures; done means test_cli_remapping passes reliably in the affected nightly jobs.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
cli, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.