Parameterized `launch_pytest` tests don't close node log files
@wjwwood is already working on this.
Since Aug 22, 2024.
- Dominant language
- Python
- Stars
- 155
- Forks
- 182
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 6
Description
Bug report
When @pytest.mark.parametrize is used in a function scope with launch_pytest the node log files from each test parameterization remain open even when execution of that specific parameterized test is complete. For large projects with dozens or hundreds of nodes this eventually causes a OSError: [Errno 24] Too many open files error.
Required Info:
- Operating System: Ubuntu Linux 20.04 amd64
- Installation type: source
- Version or commit hash: 9e517a3abd016367244fff9fa96ac8eae371a00b
- DDS implementation: RTI Connext
- Client library (if applicable): rclpy
Steps to reproduce issue
The code below cannot be run directly, but illustrates the issue. The TestContext object represents a specific test setup we want to run, and is constructed by name from a list of acceptable names in ALL_TEST_NAMES. This allows us to cherry-pick a subset of tests to run from our entire suite, with pytest -k or pytest --regex. The idea is each tests is run in sequence, against a new execution context and saved the results to a TEST_RESULTS data structure in the global context. When the test session ends, the AnalyzeResults function is called to analyze all results from all tests.
import pytest
import launch
import launch_pytest
from my_project.test_framework import TestContext, ALL_TEST_NAMES, AnalyzeResults
TEST_RESULTS = []
@pytest.fixture(scope='function')
def test_context(request):
test_context = TestContext(launch_test=request.param)
test_context.do_some_preprocessing_steps()
return test_context
@launch_pytest.fixture(scope='function')
def launch_description(test_context):
return launch.LaunchDescription(
test_context.get_all_actions() + [launch_pytest.actions.ReadyToTest()])
@pytest.mark.parametrize('test_context', ALL_TEST_NAMES, indirect=True)
@pytest.mark.launch(fixture=launch_description)
def test_execution(test_context):
results = test_context.do_some_post_postprocessing_steps()
TEST_RESULTS.append(results)
@pytest.fixture(scope='session', autouse=True)
def run_assertions():
yield
AnalyzeResults(TEST_RESULTS)
Expected behavior
Test completes without throwing an error.
Actual behavior
Python throws an internal error OSError: [Errno 24] Too many open files.
This only happens several tests into the sequence, pointing to the fact that something internally is not cleaning up after itself. I have painfully tracked lsof -p <pid_of_pytest_invocation> over time and realized that the ROS2 node log files created by each parameterized test are not being closed, even though the stack has shut down. So, let's say we have k nodes and therefore k log files created by ROS per launch test that is run. If I run n parameterized tests sequentially the number of log files open at test i, where i is in {1 ... n}, will be i * k.
Additional information
You can always increase the number of files that may be opened at once with ulimit but this is a bandaid and not a solution.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.