prometheus / prometheus/client_python

Use of `MmapedDict` results in fd leaks (resource leaks)

Đang mở
#1,008 2 bình luận 0 reaction 0 người được giao Xem trên GitHub

Chưa có ai nhận issue này.

Ngôn ngữ chính
Python
Star
4.4k
Fork
876
Merge trung bình
8 ngày 4 giờ
Pull request đã merge (30 ngày)
1

Mô tả

When running the test suite with -Wdefault, I get multiple reports of resource leaks, plus one of the tests is flaky and sometimes fails because of ResourceWarning being emitted where UserWarning was expected first:

$ python -m pytest -Wdefault
========================================================= test session starts =========================================================
platform linux -- Python 3.11.8, pytest-8.0.0, pluggy-1.4.0
rootdir: /tmp/client_python
collected 309 items                                                                                                                   

tests/openmetrics/test_exposition.py ....................                                                                       [  6%]
tests/openmetrics/test_parser.py .............................................                                                  [ 21%]
tests/test_asgi.py ssssssss                                                                                                     [ 23%]
tests/test_core.py ....................................................................................................         [ 55%]
tests/test_exposition.py ..........................................................                                             [ 74%]
tests/test_gc_collector.py ..                                                                                                   [ 75%]
tests/test_graphite_bridge.py .......                                                                                           [ 77%]
tests/test_multiprocess.py .....................F.......                                                                        [ 87%]
tests/test_parser.py ........................                                                                                   [ 94%]
tests/test_platform_collector.py ..                                                                                             [ 95%]
tests/test_process_collector.py ....                                                                                            [ 96%]
tests/test_samples.py ..                                                                                                        [ 97%]
tests/test_twisted.py s                                                                                                         [ 97%]
tests/test_wsgi.py .......                                                                                                      [100%]

============================================================== FAILURES ===============================================================
_____________________________________________ TestMultiProcess.test_remove_clear_warning ______________________________________________

self = <tests.test_multiprocess.TestMultiProcess testMethod=test_remove_clear_warning>

    def test_remove_clear_warning(self):
        os.environ['PROMETHEUS_MULTIPROC_DIR'] = self.tempdir
        with warnings.catch_warnings(record=True) as w:
            values.ValueClass = get_value_class()
            registry = CollectorRegistry()
            collector = MultiProcessCollector(registry)
            counter = Counter('c', 'help', labelnames=['label'], registry=None)
            counter.labels('label').inc()
            counter.remove('label')
            counter.clear()
            assert os.environ['PROMETHEUS_MULTIPROC_DIR'] == self.tempdir
>           assert issubclass(w[0].category, UserWarning)
E           AssertionError: assert False
E            +  where False = issubclass(<class 'ResourceWarning'>, UserWarning)
E            +    where <class 'ResourceWarning'> = <warnings.WarningMessage object at 0x7f22956b82d0>.category

tests/test_multiprocess.py:395: AssertionError
========================================================== warnings summary ===========================================================
tests/test_exposition.py::TestPushGateway::test_push_with_tls_auth_handler
  /usr/lib/python3.11/urllib/parse.py:480: ResourceWarning: unclosed <socket.socket fd=37, family=2, type=1, proto=0, laddr=('127.0.0.1', 40145)>
    for b in _UNSAFE_URL_BYTES_TO_REMOVE:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_exposition.py::TestPushGateway::test_push_with_tls_auth_handler
  /usr/lib/python3.11/urllib/parse.py:480: ResourceWarning: unclosed <socket.socket fd=38, family=2, type=1, proto=0, laddr=('127.0.0.1', 46167)>
    for b in _UNSAFE_URL_BYTES_TO_REMOVE:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_collect
  <frozen os>:676: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp9on9bz25/counter_1334413.db' mode='ab+' closefd=True>
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
  /usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpp8qzgxdb/counter_1.db' mode='ab+' closefd=True>
    with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
  /usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpp8qzgxdb/gauge_all_1.db' mode='ab+' closefd=True>
    with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
  /usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpp8qzgxdb/histogram_1.db' mode='ab+' closefd=True>
    with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
  /usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpac3ow5d8/counter_1.db' mode='ab+' closefd=True>
    with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
  /usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpac3ow5d8/gauge_all_1.db' mode='ab+' closefd=True>
    with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
  /usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpac3ow5d8/histogram_1.db' mode='ab+' closefd=True>
    with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
  /usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpx3sjgfk6/counter_1.db' mode='ab+' closefd=True>
    with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
  /usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpq0v2xj70/counter_123.db' mode='ab+' closefd=True>
    with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_all
  /usr/lib/python3.11/glob.py:176: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpq0v2xj70/counter_456.db' mode='ab+' closefd=True>
    with contextlib.closing(_iterdir(dirname, dir_fd, dironly)) as it:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
  /tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpoa1u3zei/gauge_liveall_123.db' mode='ab+' closefd=True>
    with open(filename, 'rb') as infp:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
  /tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpoa1u3zei/gauge_liveall_456.db' mode='ab+' closefd=True>
    with open(filename, 'rb') as infp:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
  /tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpkwjjpl18/gauge_livemax_123.db' mode='ab+' closefd=True>
    with open(filename, 'rb') as infp:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
  /tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpkwjjpl18/gauge_livemax_456.db' mode='ab+' closefd=True>
    with open(filename, 'rb') as infp:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
  /tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp02r2ouao/gauge_livemin_123.db' mode='ab+' closefd=True>
    with open(filename, 'rb') as infp:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
  /tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp02r2ouao/gauge_livemin_456.db' mode='ab+' closefd=True>
    with open(filename, 'rb') as infp:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
  /tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpryrzvlpo/gauge_livemostrecent_123.db' mode='ab+' closefd=True>
    with open(filename, 'rb') as infp:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
  /tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpryrzvlpo/gauge_livemostrecent_456.db' mode='ab+' closefd=True>
    with open(filename, 'rb') as infp:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
  /tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpribv4xj9/gauge_livesum_123.db' mode='ab+' closefd=True>
    with open(filename, 'rb') as infp:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
  /tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpribv4xj9/gauge_livesum_456.db' mode='ab+' closefd=True>
    with open(filename, 'rb') as infp:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
  /tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp22gzxk2n/gauge_max_123.db' mode='ab+' closefd=True>
    with open(filename, 'rb') as infp:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
  /tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp22gzxk2n/gauge_max_456.db' mode='ab+' closefd=True>
    with open(filename, 'rb') as infp:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
  /tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp6n8vag2o/gauge_min_123.db' mode='ab+' closefd=True>
    with open(filename, 'rb') as infp:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMultiProcess::test_gauge_mostrecent
  /tmp/client_python/prometheus_client/mmap_dict.py:86: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp6n8vag2o/gauge_min_456.db' mode='ab+' closefd=True>
    with open(filename, 'rb') as infp:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMmapedDict::test_expansion
  /tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/unittest.py:211: ResourceWarning: unclosed file <_io.BufferedRandom name='/tmp/tmptd9lerf6'>
    self._obj = None
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMmapedDict::test_multi_expansion
  /tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/unittest.py:211: ResourceWarning: unclosed file <_io.BufferedRandom name='/tmp/tmp4ho56w3t'>
    self._obj = None
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMmapedDict::test_process_restart
  /tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/unittest.py:211: ResourceWarning: unclosed file <_io.BufferedRandom name='/tmp/tmpp3siphvy'>
    self._obj = None
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_multiprocess.py::TestMmapedDict::test_process_restart
  /tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py:546: ResourceWarning: unclosed file <_io.BufferedRandom name='/tmp/tmpqi5kklpc'>
    fin()
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_parser.py::TestParse::test_untyped
  /tmp/client_python/tests/test_parser.py:18: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp8lpdamo1/summary_123.db' mode='ab+' closefd=True>
    for sa, sb in zip(a.samples, b.samples):
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/test_parser.py::TestParse::test_untyped
  /tmp/client_python/tests/test_parser.py:18: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp8lpdamo1/summary_456.db' mode='ab+' closefd=True>
    for sa, sb in zip(a.samples, b.samples):
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================= short test summary info =======================================================
FAILED tests/test_multiprocess.py::TestMultiProcess::test_remove_clear_warning - AssertionError: assert False
======================================== 1 failed, 299 passed, 9 skipped, 32 warnings in 4.10s ========================================

An example tracemalloc output:

$ PYTHONTRACEMALLOC=20 python -m pytest -Wdefault
[…]
tests/test_parser.py::TestParse::test_untyped
  /usr/lib/python3.11/typing.py:362: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpyaxyl9i6/summary_456.db' mode='ab+' closefd=True>
    return cached(*args, **kwds)
  
  Object allocated at:
    File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 114
      runtestprotocol(item, nextitem=nextitem)
    File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 133
      reports.append(call_and_report(item, "call", log))
    File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 226
      call = call_runtest_hook(item, when, **kwds)
    File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 265
      return CallInfo.from_call(
    File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 345
      result: Optional[TResult] = func()
    File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 266
      lambda: ihook(item=item, **kwds), when=when, reraise=reraise
    File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/pluggy/_hooks.py", line 501
      return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
    File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/pluggy/_manager.py", line 119
      return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
    File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/pluggy/_callers.py", line 102
      res = hook_impl.function(*args)
    File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/runner.py", line 173
      item.runtest()
    File "/tmp/client_python/.tox/py311/lib/python3.11/site-packages/_pytest/unittest.py", line 333
      self._testcase(result=self)  # type: ignore[arg-type]
    File "/usr/lib/python3.11/unittest/case.py", line 678
      return self.run(*args, **kwds)
    File "/usr/lib/python3.11/unittest/case.py", line 623
      self._callTestMethod(testMethod)
    File "/usr/lib/python3.11/unittest/case.py", line 579
      if method() is not None:
    File "/tmp/client_python/tests/test_multiprocess.py", line 79
      s2 = Summary('s', 'help', registry=None)
    File "/tmp/client_python/prometheus_client/metrics.py", line 151
      self._metric_init()
    File "/tmp/client_python/prometheus_client/metrics.py", line 513
      self._count = values.ValueClass(self._type, self._name, self._name + '_count', self._labelnames,
    File "/tmp/client_python/prometheus_client/values.py", line 68
      self.__reset()
    File "/tmp/client_python/prometheus_client/values.py", line 82
      files[file_prefix] = MmapedDict(filename)
    File "/tmp/client_python/prometheus_client/mmap_dict.py", line 64
      self._f = open(filename, 'rb' if read_mode else 'a+b')

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Hướng nghiên cứu

Bắt đầu trong prometheus_client/mmap_dict.py tại đường dẫn mở tệp được báo cáo ở dòng 86, sau đó kiểm tra các trường hợp đa tiến trình trong tests/test_multiprocess.py, đặc biệt là test_gauge_mostrecent và test_remove_clear_warning. Chạy bộ kiểm thử với -Wdefault và xác minh rằng MmapedDict không còn phát ra các thông báo ResourceWarning về tệp chưa được đóng hoặc gây ra lỗi do thứ tự cảnh báo.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
python
Lĩnh vực
backend
Loại issue
Lỗi
Độ khó
3/5
Thời gian dự kiến
1-2 ngày
Mức độ hoạt động
Đình trệ
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
35/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.