python / python/cpython

itertoolsmodule: free-threading use after free bug in counting slow mode

Open
#153,981 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

extension-modules type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Bug report

Bug description:

https://github.com/python/cpython/blob/main/Modules/itertoolsmodule.c#L3663

During next we swap out the pointer for lz->long_cnt inside count_nextlong. This can cause a use after free bug as shown with a new test. Running this is TSAN mode exposes this reliably.

ThreadSanitizer can not provide additional info.
SUMMARY: ThreadSanitizer: SEGV object.c:766 in PyObject_Repr
==83915==ABORTING
[1] 83915 abort PYTHON_GIL=0 ./python.exe -m unittest -v

class TestCountConcurrent(unittest.TestCase):
    @staticmethod
    def _spin_next(it, n=2000):
        for _ in range(n):
            next(it)

    @staticmethod
    def _spin_repr(it, n=2000):
        for _ in range(n):
            repr(it)

    @threading_helper.reap_threads
    def test_repr_racing_next_fast_mode(self):
        for _ in range(10):
            it = count()
            workers = [self._spin_next] * 2 + [self._spin_repr] * 4
            threading_helper.run_concurrently(workers, args=(it,))

    @threading_helper.reap_threads
    def test_repr_racing_next_slow_mode(self):
        for _ in range(10):
            # Large count to trigger "slow mode"
            it = count(10**18, 2)
            workers = [self._spin_next] * 2 + [self._spin_repr] * 4
            threading_helper.run_concurrently(workers, args=(it,))

To fix I believe we need to hold a critical section reference in repr to fetching a reference to long_cnt

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs
  • gh-153983

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review gh-153983 first, since it is linked from this issue. Then inspect Modules/itertoolsmodule.c around count_nextlong and the repr path, and run the provided TestCountConcurrent cases under ThreadSanitizer. Done means the slow-mode repr/next race no longer triggers the reported use-after-free failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, python
Domain
backend, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.