huggingface / huggingface/datasets

Invalid YAML in README.md: unknown tag !<tag:yaml.org,2002:python/tuple>

Open
#6,919 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
22k
Forks
3.4k
Avg merge
5d 7h
Merged PRs (30d)
17

Description

### Describe the bug

I wrote a notebook to load an existing dataset, process it, and upload as a private dataset using `dataset.push_to_hub(...)` at the end. The push to hub is failing with:

```
ValueError: Invalid metadata in README.md.
- Invalid YAML in README.md: unknown tag ! (50:11)

47 | - 4
48 | - 4
49 | - 8
50 | - !!binary |
----------------^
51 | TwAAAA==
52 | '1': !!python[/object/apply](http://192.168.1.128:8888/object/apply):nump ...
```

My dataset has a `train` and `validation` dataset. These are the features:

```
{'c1': Value(dtype='string', id=None),
'c2': Value(dtype='string', id=None),
'c3': [{'value': Value(dtype='string', id=None),
'start': Value(dtype='int64', id=None),
'end': Value(dtype='int64', id=None),
'label': Value(dtype='string', id=None)}],
'c4': Value(dtype='string', id=None),
'c5': Value(dtype='string', id=None),
'c6': Value(dtype='string', id=None),
'c7': Value(dtype='string', id=None),
'c8': Sequence(feature=Value(dtype='int32', id=None), length=-1, id=None),
'c9': Sequence(feature=Value(dtype='int8', id=None), length=-1, id=None),
'c10': Sequence(feature=Value(dtype='int8', id=None), length=-1, id=None),
'labels': Sequence(feature=ClassLabel(names=['O', 'B-ABC', 'I-ABC', ...], id=None), length=-1, id=None),
'c12': Sequence(feature=Value(dtype='string', id=None), length=-1, id=None)}
```

This used to work until I decided to cast the `labels` feature to a `Sequence(ClassLabel(...))` type with:

```
ds['train'] = ds['train'].cast_column("labels", Sequence(ClassLabel(names=list(labels))))
ds['validation'] = ds['validation'].cast_column("labels", Sequence(ClassLabel(names=list(labels))))
```

### Steps to reproduce the bug

1. Start with any token classification dataset.
2. Add a `labels` column with data such as `[0,0,0,12,13,13,13,0,0]`.
3. Cast the label column from `Sequence` to `Sequence(ClassLabel))` with:

```
labels = ['O', 'B-TEST', 'I-TEST']
ds = ds.cast_column("labels", Sequence(ClassLabel(names=labels)))
```

4. Push to hub with `ds.push_to_hub("me/awesome-stuff-dataset")`

### Expected behavior

I expected `push_to_hub` to successfully push my dataset to the hub without error.

### Environment info

Python 3.11.9

datasets==2.19.1
transformers==4.41.1
PyYAML==6.0.1

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.