apache / apache/arrow

[Python][Packaging][CI] Windows wheels build output shows `ValueError: underlying buffer has been detached`

Open
#35,355 0 comments 0 reactions 0 assignees View on GitHub
Component: Continuous Integration Component: Packaging Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 18h
Merged PRs (30d)
91

Description

### Describe the bug, including details regarding any error messages, version, and platform.

Investigating some issues around windows wheels I can see an `ValueError: underlying buffer has been detached` error log repeating for several thousands of lines for the different files to the Wheel:
```

--- Logging error ---
Traceback (most recent call last):
File "C:\Python311\Lib\logging\__init__.py", line 1113, in emit
stream.write(msg + self.terminator)
ValueError: underlying buffer has been detached
Call stack:
File "C:\arrow\python\setup.py", line 474, in
setup(
File "C:\Python311\Lib\site-packages\setuptools\__init__.py", line 87, in setup
return distutils.core.setup(**attrs)
File "C:\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 185, in setup
return run_commands(dist)
File "C:\Python311\Lib\site-packages\setuptools\_distutils\core.py", line 201, in run_commands
dist.run_commands()
File "C:\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 968, in run_commands
self.run_command(cmd)
File "C:\Python311\Lib\site-packages\setuptools\dist.py", line 1217, in run_command
super().run_command(command)
File "C:\Python311\Lib\site-packages\setuptools\_distutils\dist.py", line 987, in run_command
cmd_obj.run()
File "C:\Python311\Lib\site-packages\wheel\bdist_wheel.py", line 387, in run
wf.write_files(archive_root)
File "C:\Python311\Lib\site-packages\wheel\wheelfile.py", line 137, in write_files
self.write(path, arcname)
File "C:\Python311\Lib\site-packages\wheel\wheelfile.py", line 153, in write
self.writestr(zinfo, data, compress_type)
File "C:\Python311\Lib\site-packages\wheel\wheelfile.py", line 165, in writestr
log.info(f"adding '{fname}'")
Message: "adding 'pyarrow/tests/test_cpp_internals.py'"
Arguments: ()
```
This doesn't seem to be causing any issue but it makes slightly more difficult to understand errors and issues. I am opening it so we can investigate if we can get rid of it.
This is an example of the log, that built is failed but the issue is not related to this error: https://github.com/ursacomputing/crossbow/actions/runs/4819106737/jobs/8581899869#step:7:8640

### Component(s)

Continuous Integration, Packaging, Python

Contributor guide

Open the contributing guide

Research direction

Start with the Windows wheel build invoked from python/setup.py and review the wheel bdist_wheel and WheelFile logging shown in the traceback. Use the linked Crossbow GitHub Actions run to reproduce or inspect the failure. Done means the build no longer emits repeated “underlying buffer has been detached” logging errors, while genuine build failures remain visible.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, python
Domain
build-system, ci-cd
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.