huggingface / huggingface/dataset-viewer

Remove extra `label` column

Open
#3,014 0 comments 0 reactions 0 assignees View on GitHub
blocked-by-upstream bug P2
Dominant language
Python
Stars
899
Forks
130
Avg merge
9h 26m
Merged PRs (30d)
14

Description

In example dataset https://huggingface.co/datasets/datasets-examples/doc-audio-4, we have an "unexpected" label column with only `null` values.

Capture d’écran 2024-08-02 à 12 33 10

I think it's due to a "collision" between the heuristics that define splits and/or classes based on the directories. There is a `drop_labels=True` option in the datasets library, if it helps.

Ideally, in this case, we should have two splits (train and test), and no additional `label` column.

I think the issue also exists with image datasets.

Contributor guide

Open the contributing guide

Research direction

Reproduce the behavior with the doc-audio-4 dataset and inspect the dataset-viewer path that infers splits and class labels from directories. Check how the datasets library's drop_labels=True option relates to this flow, then verify that the example exposes only train and test splits with no extra label column, including for image datasets.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.