python / python/cpython

Thread-unsafe libc functions

Open
#127,081 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

extension-modules interpreter-core topic-free-threading type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Bug report

There are a a few non thread-safe libc functions used in Python that can be an issue for free threading, isolated subinterpreters, or sometimes even with the GIL.

In https://github.com/python/cpython/issues/126316, @vstinner fixed the use setgrent / getgrent. It's probably a good time to look for other similar issues.

clang-tidy

clang-tidy has a concurrency-mt-unsafe check that looks for "known-to-be-unsafe functions".

clang-tidy notes

Prerequisites: install clang-tidy-18 and bear.

./configure -C --with-pydebug --disable-gil

# generate compile_commands.json
bear -- make -j 

run-clang-tidy-18 -checks='-*,concurrency-mt-unsafe' -p .
Unsafe libc functions
  • localeconv(): not thread-safe, see glibc's manual. Is nl_langinfo a substitue?
  • setlocale
  • setpwent, getpwent, and endpwent in pwdmodule.c. These are similar to grpmodule.c and can likely be addressed the same way.
  • getservbyname, getservbyport, getprotobyname in Modules/socketmodule.c: use getservbyname_r, etc.? Note these thread-safety issues affect the default build because we release the GIL around the relevant calls.
  • dbm_open, dbm_close, etc. in Modules/_dbmmodule.c
  • getlogin: use getlogin_r if available?
Unfixable by us?
Safe due to our usage
  • getc_unlocked, safe because we use it within a flockfile() call.
  • mbrtowc() - safe as long as the passed in mbstate_t * is non-NULL, which is the case in CPython.
Safe in glibc

These functions are flagged by clang-tidy because they are not guaranteed to be safe by POSIX, but they are safe in glibc. It'd be nice to verify that they are safe in other libc implementations. I don't think it's worth changing them:

Other
  • exit(): apparently concurrent calls to exit() are not thread-safe, but I don't think it matters for our usages.
  • ptsname(): we already use ptsname_r(), but the static analyzer gets confused
Linked PRs
  • gh-132591
  • gh-132748
  • gh-132749
  • gh-132750
  • gh-132751
  • gh-135097
  • gh-135098
  • gh-139996

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review the linked PRs first, then inspect the clang-tidy concurrency-mt-unsafe results for Modules/pwdmodule.c, Modules/socketmodule.c, and Modules/_dbmmodule.c. Run the documented clang-tidy command after configuring with --disable-gil; done means each relevant libc use is addressed or its safety is justified.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, python
Domain
operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.