[BUG] - python setup.py bdist_wheel did not run successfully - Error randomly appears during uwsgi build.

setuptools version

setuptools==70.1.1

Python version

3.12.4

OS

Linux

Additional environment information

Docker image python:3.12-bullseye@sha256:c7e1479380ded85f86dc665defb7b519fc445f93df916195f213a4440babe820
it includes default packages:
wheel 0.43.0
setuptools 70.0.0

Issue is happening both with poetry and pipenv when installing uwsgi==2.0.25.1 (and 2.0.26)

Issue is not happening with python:3.11-bullseye (defaults: wheel 0.43.0, setuptools 65.5.1) image and "setuptools==69.5.1"

Description

During docker build about half of the builds failed with this error.

Before docker build we run:
for pipenv:

RUN pip install --upgrade pip pipenv wheel

for poetry:

RUN pip install --upgrade pip setuptools \
  && pip install poetry \
  && poetry config virtualenvs.create false

Both of them result in error happening randomly and build failes.

Expected behavior

Successful build

How to Reproduce

I will try to prepare something, but as I said earlier. It's happening randomly.

Output

[...]
Building wheel for uwsgi (setup.py): started
Building wheel for uwsgi (setup.py): finished with status 'error'
Running setup.py clean for uwsgi
Successfully built coreschema django-celery-beat django-computedfields django-fast-update django-querycount openapi-codec pysftp python-crontab
Failed to build uwsgi
error: subprocess-exited-with-error
  
   × python setup.py bdist_wheel did not run successfully.
   │ exit code: 1
   ╰─> [238 lines of output]
       /usr/local/lib/python3.12/site-packages/setuptools/_distutils/dist.py:266: UserWarning: Unknown distribution option: 'descriptions'
         warnings.warn(msg)
       running bdist_wheel
       running build
       running build_py
       creating build
       creating build/lib
       copying uwsgidecorators.py -> build/lib
       warning: build_py: byte-compiling is disabled, skipping.
       
       /usr/local/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
       !!
       
               ********************************************************************************
               Please avoid running ``setup.py`` directly.
               Instead, use pypa/build, pypa/installer or other
               standards-based tools.
       
               See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
               ********************************************************************************
       
       !!
         self.initialize_options()
       installing to build/bdist.linux-x86_64/wheel
       running install
       using profile: buildconf/default.ini
       detected include path: ['/usr/lib/gcc/x86_64-linux-gnu/10/include', '/usr/local/include', '/usr/include/x86_64-linux-gnu', '/usr/include']
       Patching "bin_name" to properly install_scripts dir
       detected CPU cores: 32
       configured CFLAGS: -O2 -I. -Wall -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wformat-signedness -DUWSGI_HAS_IFADDRS -DUWSGI_ZLIB -DUWSGI_LOCK_USE_MUTEX -DUWSGI_EVENT_USE_EPOLL -DUWSGI_EVENT_TIMER_USE_TIMERFD -DUWSGI_EVENT_FILEMONITOR_USE_INOTIFY  -DUWSGI_PCRE2 -DUWSGI_ROUTING -DUWSGI_UUID -DUWSGI_VERSION="\"2.0.26\"" -DUWSGI_VERSION_BASE="2" -DUWSGI_VERSION_MAJOR="0" -DUWSGI_VERSION_MINOR="26" -DUWSGI_VERSION_REVISION="0" -DUWSGI_VERSION_CUSTOM="\"\"" -DUWSGI_YAML -DUWSGI_SSL -I/usr/include/libxml2 -DUWSGI_XML -DUWSGI_XML_LIBXML2 -DUWSGI_PLUGIN_DIR="\".\"" -DUWSGI_DECLARE_EMBEDDED_PLUGINS="UDEP(python);UDEP(gevent);UDEP(ping);UDEP(cache);UDEP(nagios);UDEP(rrdtool);UDEP(carbon);UDEP(rpc);UDEP(corerouter);UDEP(fastrouter);UDEP(http);UDEP(ugreen);UDEP(signal);UDEP(syslog);UDEP(rsyslog);UDEP(logsocket);UDEP(router_uwsgi);UDEP(router_redirect);UDEP(router_basicauth);UDEP(zergpool);UDEP(redislog);UDEP(mongodblog);UDEP(router_rewrite);UDEP(router_http);UDEP(logfile);UDEP(router_cache);UDEP(rawrouter);UDEP(router_static);UDEP(sslrouter);UDEP(spooler);UDEP(cheaper_busyness);UDEP(symcall);UDEP(transformation_tofile);UDEP(transformation_gzip);UDEP(transformation_chunked);UDEP(transformation_offload);UDEP(router_memcached);UDEP(router_redis);UDEP(router_hash);UDEP(router_expires);UDEP(router_metrics);UDEP(transformation_template);UDEP(stats_pusher_socket);" -DUWSGI_LOAD_EMBEDDED_PLUGINS="ULEP(python);ULEP(gevent);ULEP(ping);ULEP(cache);ULEP(nagios);ULEP(rrdtool);ULEP(carbon);ULEP(rpc);ULEP(corerouter);ULEP(fastrouter);ULEP(http);ULEP(ugreen);ULEP(signal);ULEP(syslog);ULEP(rsyslog);ULEP(logsocket);ULEP(router_uwsgi);ULEP(router_redirect);ULEP(router_basicauth);ULEP(zergpool);ULEP(redislog);ULEP(mongodblog);ULEP(router_rewrite);ULEP(router_http);ULEP(logfile);ULEP(router_cache);ULEP(rawrouter);ULEP(router_static);ULEP(sslrouter);ULEP(spooler);ULEP(cheaper_busyness);ULEP(symcall);ULEP(transformation_tofile);ULEP(transformation_gzip);ULEP(transformation_chunked);ULEP(transformation_offload);ULEP(router_memcached);ULEP(router_redis);ULEP(router_hash);ULEP(router_expires);ULEP(router_metrics);ULEP(transformation_template);ULEP(stats_pusher_socket);"
       *** uWSGI compiling server core ***
       [thread 0][gcc -pthread] core/utils.o
       [thread 2][gcc -pthread] core/protocol.o
       [thread 3][gcc -pthread] core/socket.o
       [thread 4][gcc -pthread] core/logging.o
       [thread 5][gcc -pthread] core/master.o
       [thread 7][gcc -pthread] core/master_utils.o
       [thread 6][gcc -pthread] core/emperor.o
       [thread 8][gcc -pthread] core/notify.o
       [thread 9][gcc -pthread] core/mule.o
       [thread 11][gcc -pthread] core/subscription.o
       [thread 10][gcc -pthread] core/stats.o
       [thread 12][gcc -pthread] core/sendfile.o
       [thread 14][gcc -pthread] core/async.o
       [thread 13][gcc -pthread] core/master_checks.o
       [thread 15][gcc -pthread] core/fifo.o
       [thread 16][gcc -pthread] core/offload.o
       [thread 17][gcc -pthread] core/io.o
       [thread 18][gcc -pthread] core/static.o
       [thread 19][gcc -pthread] core/websockets.o
       [thread 21][gcc -pthread] core/spooler.o
       [thread 20][gcc -pthread] core/snmp.o
       [thread 22][gcc -pthread] core/exceptions.o
       [thread 23][gcc -pthread] core/config.o
       [thread 25][gcc -pthread] core/setup_utils.o
       [thread 24][gcc -pthread] core/clock.o
       [thread 26][gcc -pthread] core/init.o
       [thread 28][gcc -pthread] core/buffer.o
       [thread 27][gcc -pthread] core/reader.o
       [thread 30][gcc -pthread] core/writer.o
       [thread 29][gcc -pthread] core/alarm.o
       [thread 31][gcc -pthread] core/cron.o
       [thread 1][gcc -pthread] core/hooks.o
       [thread 12][gcc -pthread] core/plugins.o
       [thread 8][gcc -pthread] core/lock.o
       [thread 25][gcc -pthread] core/cache.o
       [thread 24][gcc -pthread] core/daemons.o
       [thread 15][gcc -pthread] core/errors.o
       [thread 20][gcc -pthread] core/hash.o
       [thread 31][gcc -pthread] core/master_events.o
       [thread 26][gcc -pthread] core/chunked.o
       [thread 29][gcc -pthread] core/queue.o
       [thread 22][gcc -pthread] core/event.o
       [thread 28][gcc -pthread] core/signal.o
       [thread 19][gcc -pthread] core/strings.o
       [thread 13][gcc -pthread] core/progress.o
       [thread 14][gcc -pthread] core/timebomb.o
       [thread 21][gcc -pthread] core/ini.o
       Exception in thread Thread-15 (thread_compiler):
       Traceback (most recent call last):
         File "/usr/local/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
       [thread 9][gcc -pthread] core/fsmon.o
           self.run()
         File "/usr/local/lib/python3.12/threading.py", line 1010, in run
           self._target(*self._args, **self._kwargs)
         File "/tmp/pip-install-zbz50gqu/uwsgi_86523fde0bc04ec093172a1cc8f60478/uwsgiconfig.py", line 116, in thread_compiler
           ret = subprocess.call(cmdline, shell=True)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
         File "/usr/local/lib/python3.12/subprocess.py", line 389, in call
       [thread 16][gcc -pthread] core/mount.o
       [thread 1][gcc -pthread] core/metrics.o
           with Popen(*popenargs, **kwargs) as p:
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
         File "/usr/local/lib/python3.12/subprocess.py", line 1026, in __init__
           self._execute_child(args, executable, preexec_fn, close_fds,
         File "/usr/local/lib/python3.12/subprocess.py", line 1955, in _execute_child
           raise child_exception_type(errno_num, err_msg, err_filename)
       OSError: [Errno 14] Bad address: '/bin/sh'
       [thread 18][gcc -pthread] core/plugins_builder.o
       [thread 27][gcc -pthread] core/sharedarea.o
       [thread 10][gcc -pthread] core/rpc.o
       [thread 30][gcc -pthread] core/gateway.o
       [thread 23][gcc -pthread] core/loop.o
       [thread 2][gcc -pthread] core/cookie.o
       [thread 11][gcc -pthread] core/querystring.o
       [thread 20][gcc -pthread] core/rb_timers.o
       [thread 5][gcc -pthread] core/transformations.o
       [thread 15][gcc -pthread] core/uwsgi.o
       [thread 31][gcc -pthread] proto/uwsgi.o
       [thread 12][gcc -pthread] proto/base.o
       [thread 8][gcc -pthread] proto/http.o
       [thread 29][gcc -pthread] proto/fastcgi.o
       [thread 26][gcc -pthread] proto/scgi.o
       [thread 17][gcc -pthread] proto/puwsgi.o
       [thread 13][gcc -pthread] lib/linux_ns.o
       [thread 24][gcc -pthread] core/zlib.o
       [thread 21][gcc -pthread] core/regexp.o
       [thread 18][gcc -pthread] core/routing.o
       [thread 9][gcc -pthread] core/yaml.o
       [thread 22][gcc -pthread] core/ssl.o
       [thread 16][gcc -pthread] core/legion.o
       [thread 19][gcc -pthread] core/xmlconf.o
       [thread 10][gcc -pthread] core/dot_h.o
       [thread 28][gcc -pthread] core/config_py.o
       *** uWSGI compiling embedded plugins ***
       [thread 30][gcc -pthread] plugins/python/python_plugin.o
       [thread 4][gcc -pthread] plugins/python/pyutils.o
       [thread 11][gcc -pthread] plugins/python/pyloader.o
       [thread 7][gcc -pthread] plugins/python/wsgi_handlers.o
       [thread 2][gcc -pthread] plugins/python/wsgi_headers.o
       [thread 28][gcc -pthread] plugins/python/wsgi_subhandler.o
       [thread 23][gcc -pthread] plugins/python/web3_subhandler.o
       [thread 3][gcc -pthread] plugins/python/pump_subhandler.o
       [thread 10][gcc -pthread] plugins/python/gil.o
       [thread 5][gcc -pthread] plugins/python/uwsgi_pymodule.o
       [thread 20][gcc -pthread] plugins/python/profiler.o
       [thread 27][gcc -pthread] plugins/python/symimporter.o
       [thread 31][gcc -pthread] plugins/python/tracebacker.o
       [thread 26][gcc -pthread] plugins/python/raw.o
       [thread 17][gcc -pthread] plugins/gevent/gevent.o
      [thread 6][gcc -pthread] plugins/gevent/hooks.o
       [thread 29][gcc -pthread] plugins/ping/ping_plugin.o
       [thread 24][gcc -pthread] plugins/cache/cache.o
       [thread 21][gcc -pthread] plugins/nagios/nagios.o
       [thread 13][gcc -pthread] plugins/rrdtool/rrdtool.o
       [thread 12][gcc -pthread] plugins/carbon/carbon.o
       [thread 9][gcc -pthread] plugins/rpc/rpc_plugin.o
       plugins/python/pyutils.c: In function ‘init_pyargv’:
       plugins/python/pyutils.c:391:2: warning: ‘PySys_SetArgv’ is deprecated [-Wdeprecated-declarations]
         391 |  PySys_SetArgv(up.argc, up.py_argv);
             |  ^~~~~~~~~~~~~
       In file included from /usr/local/include/python3.12/Python.h:96,
                        from plugins/python/uwsgi_python.h:4,
                        from plugins/python/pyutils.c:1:
       /usr/local/include/python3.12/sysmodule.h:13:38: note: declared here
          13 | Py_DEPRECATED(3.11) PyAPI_FUNC(void) PySys_SetArgv(int, wchar_t **);
             |                                      ^~~~~~~~~~~~~
       [thread 19][gcc -pthread] plugins/corerouter/cr_common.o
       plugins/python/python_plugin.c:138:2: warning: ‘Py_NoSiteFlag’ is deprecated [-Wdeprecated-declarations]
         138 |  {"no-site", no_argument, 0, "do not import site module", uwsgi_opt_true, &Py_NoSiteFlag, 0},
             |  ^
       In file included from /usr/local/include/python3.12/Python.h:48,
                        from plugins/python/uwsgi_python.h:4,
                        from plugins/python/python_plugin.c:1:
       /usr/local/include/python3.12/cpython/pydebug.h:14:37: note: declared here
          14 | Py_DEPRECATED(3.12) PyAPI_DATA(int) Py_NoSiteFlag;
             |                                     ^~~~~~~~~~~~~
       plugins/python/python_plugin.c: In function ‘uwsgi_python_init’:
       plugins/python/python_plugin.c:254:3: warning: ‘Py_SetPythonHome’ is deprecated [-Wdeprecated-declarations]
         254 |   Py_SetPythonHome(wpyhome);
             |   ^~~~~~~~~~~~~~~~
       In file included from /usr/local/include/python3.12/Python.h:94,
                        from plugins/python/uwsgi_python.h:4,
                        from plugins/python/python_plugin.c:1:
       /usr/local/include/python3.12/pylifecycle.h:40:38: note: declared here
          40 | Py_DEPRECATED(3.11) PyAPI_FUNC(void) Py_SetPythonHome(const wchar_t *);
             |                                      ^~~~~~~~~~~~~~~~
       plugins/python/python_plugin.c:278:2: warning: ‘Py_SetProgramName’ is deprecated [-Wdeprecated-declarations]
         278 |  Py_SetProgramName(pname);
             |  ^~~~~~~~~~~~~~~~~
       In file included from /usr/local/include/python3.12/Python.h:94,
                        from plugins/python/uwsgi_python.h:4,
                        from plugins/python/python_plugin.c:1:
       /usr/local/include/python3.12/pylifecycle.h:37:38: note: declared here
          37 | Py_DEPRECATED(3.11) PyAPI_FUNC(void) Py_SetProgramName(const wchar_t *);
             |                                      ^~~~~~~~~~~~~~~~~
       plugins/python/python_plugin.c:287:2: warning: ‘Py_OptimizeFlag’ is deprecated [-Wdeprecated-declarations]
         287 |  Py_OptimizeFlag = up.optimize;
             |  ^~~~~~~~~~~~~~~
       In file included from /usr/local/include/python3.12/Python.h:48,
                        from plugins/python/uwsgi_python.h:4,
                        from plugins/python/python_plugin.c:1:
       /usr/local/include/python3.12/cpython/pydebug.h:13:37: note: declared here
          13 | Py_DEPRECATED(3.12) PyAPI_DATA(int) Py_OptimizeFlag;
             |                                     ^~~~~~~~~~~~~~~
       [thread 22][gcc -pthread] plugins/corerouter/cr_map.o
       [thread 1][gcc -pthread] plugins/corerouter/corerouter.o
       [thread 25][gcc -pthread] plugins/fastrouter/fastrouter.o
       [thread 10][gcc -pthread] plugins/http/http.o
       [thread 2][gcc -pthread] plugins/http/keepalive.o
       [thread 8][gcc -pthread] plugins/http/https.o
       [thread 23][gcc -pthread] plugins/http/spdy3.o
       [thread 4][gcc -pthread] plugins/ugreen/ugreen.o
       [thread 20][gcc -pthread] plugins/signal/signal_plugin.o
       [thread 28][gcc -pthread] plugins/syslog/syslog_plugin.o
       [thread 26][gcc -pthread] plugins/rsyslog/rsyslog_plugin.o
       [thread 29][gcc -pthread] plugins/logsocket/logsocket_plugin.o
       [thread 7][gcc -pthread] plugins/router_uwsgi/router_uwsgi.o
       [thread 21][gcc -pthread] plugins/router_redirect/router_redirect.o
       [thread 3][gcc -pthread] plugins/router_basicauth/router_basicauth.o
       [thread 24][gcc -pthread] plugins/zergpool/zergpool.o
       [thread 13][gcc -pthread] plugins/redislog/redislog_plugin.o
       [thread 31][gcc -pthread] plugins/mongodblog/mongodblog_plugin.o
       [thread 27][gcc -pthread] plugins/router_rewrite/router_rewrite.o
       [thread 11][gcc -pthread] plugins/router_http/router_http.o
       [thread 16][gcc -pthread] plugins/logfile/logfile.o
       [thread 12][gcc -pthread] plugins/router_cache/router_cache.o
       [thread 6][gcc -pthread] plugins/rawrouter/rawrouter.o
       [thread 22][gcc -pthread] plugins/router_static/router_static.o
       [thread 17][gcc -pthread] plugins/sslrouter/sslrouter.o
       [thread 19][gcc -pthread] plugins/spooler/spooler_plugin.o
       [thread 20][gcc -pthread] plugins/cheaper_busyness/cheaper_busyness.o
       [thread 2][gcc -pthread] plugins/symcall/symcall_plugin.o
       [thread 4][gcc -pthread] plugins/transformation_tofile/tofile.o
       [thread 28][gcc -pthread] plugins/transformation_gzip/gzip.o
       [thread 26][gcc -pthread] plugins/transformation_chunked/chunked.o
       [thread 29][gcc -pthread] plugins/transformation_offload/offload.o
       [thread 9][gcc -pthread] plugins/router_memcached/router_memcached.o
       [thread 0][gcc -pthread] plugins/router_redis/router_redis.o
       [thread 21][gcc -pthread] plugins/router_hash/router_hash.o
       [thread 7][gcc -pthread] plugins/router_expires/expires.o
       [thread 25][gcc -pthread] plugins/router_metrics/plugin.o
       [thread 24][gcc -pthread] plugins/transformation_template/tt.o
       [thread 31][gcc -pthread] plugins/stats_pusher_socket/plugin.o
       *** uWSGI linking ***
       gcc -pthread -o build/bdist.linux-x86_64/wheel/uWSGI-2.0.26.data/scripts/uwsgi -L/usr/local/lib -Wl,-rpath,/usr/local/lib core/utils.o core/protocol.o core/socket.o core/logging.o core/master.o core/master_utils.o core/emperor.o core/notify.o core/mule.o core/subscription.o core/stats.o core/sendfile.o core/async.o core/master_checks.o core/fifo.o core/offload.o core/io.o core/static.o core/websockets.o core/spooler.o core/snmp.o core/exceptions.o core/config.o core/setup_utils.o core/clock.o core/init.o core/buffer.o core/reader.o core/writer.o core/alarm.o core/cron.o core/hooks.o core/plugins.o core/lock.o core/cache.o core/daemons.o core/errors.o core/hash.o core/master_events.o core/chunked.o core/queue.o core/event.o core/signal.o core/strings.o core/progress.o core/timebomb.o core/ini.o core/fsmon.o core/mount.o core/metrics.o core/plugins_builder.o core/sharedarea.o core/rpc.o core/gateway.o core/loop.o core/cookie.o core/querystring.o core/rb_timers.o core/transformations.o core/uwsgi.o proto/base.o proto/uwsgi.o proto/http.o proto/fastcgi.o proto/scgi.o proto/puwsgi.o lib/linux_ns.o core/zlib.o core/regexp.o core/routing.o core/yaml.o core/ssl.o core/legion.o core/xmlconf.o core/dot_h.o core/config_py.o plugins/python/python_plugin.o plugins/python/pyutils.o plugins/python/pyloader.o plugins/python/wsgi_handlers.o plugins/python/wsgi_headers.o plugins/python/wsgi_subhandler.o plugins/python/web3_subhandler.o plugins/python/pump_subhandler.o plugins/python/gil.o plugins/python/uwsgi_pymodule.o plugins/python/profiler.o plugins/python/symimporter.o plugins/python/tracebacker.o plugins/python/raw.o plugins/gevent/gevent.o plugins/gevent/hooks.o plugins/ping/ping_plugin.o plugins/cache/cache.o plugins/nagios/nagios.o plugins/rrdtool/rrdtool.o plugins/carbon/carbon.o plugins/rpc/rpc_plugin.o plugins/corerouter/cr_common.o plugins/corerouter/cr_map.o plugins/corerouter/corerouter.o plugins/fastrouter/fastrouter.o plugins/http/http.o plugins/http/keepalive.o plugins/http/https.o plugins/http/spdy3.o plugins/ugreen/ugreen.o plugins/signal/signal_plugin.o plugins/syslog/syslog_plugin.o plugins/rsyslog/rsyslog_plugin.o plugins/logsocket/logsocket_plugin.o plugins/router_uwsgi/router_uwsgi.o plugins/router_redirect/router_redirect.o plugins/router_basicauth/router_basicauth.o plugins/zergpool/zergpool.o plugins/redislog/redislog_plugin.o plugins/mongodblog/mongodblog_plugin.o plugins/router_rewrite/router_rewrite.o plugins/router_http/router_http.o plugins/logfile/logfile.o plugins/router_cache/router_cache.o plugins/rawrouter/rawrouter.o plugins/router_static/router_static.o plugins/sslrouter/sslrouter.o plugins/spooler/spooler_plugin.o plugins/cheaper_busyness/cheaper_busyness.o plugins/symcall/symcall_plugin.o plugins/transformation_tofile/tofile.o plugins/transformation_gzip/gzip.o plugins/transformation_chunked/chunked.o plugins/transformation_offload/offload.o plugins/router_memcached/router_memcached.o plugins/router_redis/router_redis.o plugins/router_hash/router_hash.o plugins/router_expires/expires.o plugins/router_metrics/plugin.o plugins/transformation_template/tt.o plugins/stats_pusher_socket/plugin.o -lpthread -lm -rdynamic -ldl -lz -lpcre2-8 -luuid -lssl -lcrypto -lxml2 -lpthread -ldl -lutil -lm -lpython3.12 -lcrypt
       gcc: error: core/timebomb.o: No such file or directory
       *** error linking uWSGI ***
       [end of output]
   
   note: This error originates from a subprocess, and is likely not a problem with pip.
   ERROR: Failed building wheel for uwsgi
 ERROR: Could not build wheels for uwsgi, which is required to install pyproject.toml-based projects
ERROR: Couldn't install package: {}
Package installation failed...

Start by reproducing the Python 3.12 Linux build of uwsgi with setuptools 70.1.1 and inspect uwsgiconfig.py around thread_compiler at line 116, where the /bin/sh OSError occurs. Compare the intermittent failure with the successful Python 3.11 and setuptools 69.5.1 builds; done means identifying a reliable cause and a confirmed setuptools-side change or clear limitation.