huggingface / huggingface/datasets

.map() fails if function uses pyvista

Open
#7,512 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
22k
Forks
3.4k
Avg merge
5d 7h
Merged PRs (30d)
17

Description

### Describe the bug

Using PyVista inside a .map() produces a crash with `objc[78796]: +[NSResponder initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.`

### Steps to reproduce the bug

Run

```python
import numpy as np
import pyvista as pv
import datasets

data = [{"coords": np.random.rand(5, 3)} for _ in range(3)]

def render_point(example):
plotter = pv.Plotter(off_screen=True)
cloud = pv.PolyData(example["coords"])
plotter.add_mesh(cloud)
img = plotter.screenshot(return_img=True)
return {"image": img}

# breaks if num_proc>1
ds = datasets.Dataset.from_list(data).map(render_point, num_proc=2)

```

### Expected behavior

It should work. Just like when I use a process pool to make it work.

```python
import numpy as np
import pyvista as pv
import multiprocessing

data = [{"coords": np.random.rand(5, 3)} for _ in range(3)]

def render_point(example):
plotter = pv.Plotter(off_screen=True)
cloud = pv.PolyData(example["coords"])
plotter.add_mesh(cloud)
img = plotter.screenshot(return_img=True)
return {"image": img}

if __name__ == "__main__":
with multiprocessing.Pool(processes=2) as pool:
results = pool.map(render_point, data)
print(results[0]["image"].shape)
```

### Environment info

- `datasets` version: 3.3.2
- Platform: macOS-15.3.2-arm64-arm-64bit
- Python version: 3.11.10
- `huggingface_hub` version: 0.28.1
- PyArrow version: 18.1.0
- Pandas version: 2.2.3
- `fsspec` version: 2024.10.0

And then I suppose pyvista info is good to have.

```python
import pyvista as pv
print(pv.Report())
```
gives
--------------------------------------------------------------------------------
Date: Mon Apr 14 21:42:08 2025 CEST

OS : Darwin (macOS 15.3.2)
CPU(s) : 10
Machine : arm64
Architecture : 64bit
RAM : 32.0 GiB
Environment : IPython
File system : apfs
GPU Vendor : Apple
GPU Renderer : Apple M1 Max
GPU Version : 4.1 Metal - 89.3
MathText Support : True

Python 3.11.10 (main, Oct 7 2024, 23:25:27) [Clang 18.1.8 ]

pyvista : 0.44.2
vtk : 9.4.0
numpy : 2.2.2
matplotlib : 3.10.0
scooby : 0.10.0
pooch : 1.8.2
pillow : 11.1.0
imageio : 2.36.1
PyQt5 : 5.15.11
IPython : 8.30.0
scipy : 1.14.1
tqdm : 4.67.1
jupyterlab : 4.3.5
nest_asyncio : 1.6.0
--------------------------------------------------------------------------------

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.