huggingface / huggingface/datasets
.map() fails if function uses pyvista
- Dominant language
- Python
- Stars
- 22k
- Forks
- 3.4k
- Avg merge
- 5d 7h
- Merged PRs (30d)
- 17
Description
### Describe the bug
Using PyVista inside a .map() produces a crash with `objc[78796]: +[NSResponder initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.`
### Steps to reproduce the bug
Run
```python
import numpy as np
import pyvista as pv
import datasets
data = [{"coords": np.random.rand(5, 3)} for _ in range(3)]
def render_point(example):
plotter = pv.Plotter(off_screen=True)
cloud = pv.PolyData(example["coords"])
plotter.add_mesh(cloud)
img = plotter.screenshot(return_img=True)
return {"image": img}
# breaks if num_proc>1
ds = datasets.Dataset.from_list(data).map(render_point, num_proc=2)
```
### Expected behavior
It should work. Just like when I use a process pool to make it work.
```python
import numpy as np
import pyvista as pv
import multiprocessing
data = [{"coords": np.random.rand(5, 3)} for _ in range(3)]
def render_point(example):
plotter = pv.Plotter(off_screen=True)
cloud = pv.PolyData(example["coords"])
plotter.add_mesh(cloud)
img = plotter.screenshot(return_img=True)
return {"image": img}
if __name__ == "__main__":
with multiprocessing.Pool(processes=2) as pool:
results = pool.map(render_point, data)
print(results[0]["image"].shape)
```
### Environment info
- `datasets` version: 3.3.2
- Platform: macOS-15.3.2-arm64-arm-64bit
- Python version: 3.11.10
- `huggingface_hub` version: 0.28.1
- PyArrow version: 18.1.0
- Pandas version: 2.2.3
- `fsspec` version: 2024.10.0
And then I suppose pyvista info is good to have.
```python
import pyvista as pv
print(pv.Report())
```
gives
--------------------------------------------------------------------------------
Date: Mon Apr 14 21:42:08 2025 CEST
OS : Darwin (macOS 15.3.2)
CPU(s) : 10
Machine : arm64
Architecture : 64bit
RAM : 32.0 GiB
Environment : IPython
File system : apfs
GPU Vendor : Apple
GPU Renderer : Apple M1 Max
GPU Version : 4.1 Metal - 89.3
MathText Support : True
Python 3.11.10 (main, Oct 7 2024, 23:25:27) [Clang 18.1.8 ]
pyvista : 0.44.2
vtk : 9.4.0
numpy : 2.2.2
matplotlib : 3.10.0
scooby : 0.10.0
pooch : 1.8.2
pillow : 11.1.0
imageio : 2.36.1
PyQt5 : 5.15.11
IPython : 8.30.0
scipy : 1.14.1
tqdm : 4.67.1
jupyterlab : 4.3.5
nest_asyncio : 1.6.0
--------------------------------------------------------------------------------
Contributor guide
Assessment
This issue has not been assessed yet.