huggingface / huggingface/datasets

TypeError in _dill.py _batch_setitems with Python 3.14

Open
#8,373 8 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
22k
Forks
3.4k
Avg merge
5d 7h
Merged PRs (30d)
17

Description

### Describe the bug

When using `load_dataset` on Python 3.14, a `TypeError` is raised during the fingerprinting process inside the bundled `_dill.py`. This happens because Python 3.14 changed the signature of the internal C-level `_Pickler._batch_setitems` to require 3 arguments (`self, items, obj`), while `datasets/utils/_dill.py` only passes 2 arguments.

### Steps to reproduce the bug

Run any `load_dataset` script on Python 3.14:
```python
from datasets import load_dataset
ds = load_dataset('parquet', data_files={'train': 'data.parquet'})

Error output:

TypeError: _Pickler._batch_setitems() missing 1 required positional argument: 'obj'

### Expected behavior

That this thing works (?)

### Environment info

nope

Contributor guide

Open the contributing guide

Research direction

Start in datasets/utils/_dill.py and reproduce the failure by running the reported load_dataset example on Python 3.14. Trace the fingerprinting call into _Pickler._batch_setitems and verify the compatibility behavior; done means the example completes without the TypeError on Python 3.14.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.