huggingface / huggingface/aisheets

ValueError: Columns defined in both source dataset and generation config

Open
#444 7 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
1.6k
Forks
140
Avg merge
1m
Merged PRs (30d)
1

Description

I am trying to create a large dataset with the script provided in the README.md. Here is my script:

```python
hf jobs uv run --namespace $NAMESPACE \
-s HF_TOKEN=$HF_TOKEN \
https://github.com/huggingface/aisheets/raw/refs/heads/main/scripts/extend_dataset/with_inference_client.py \
snork-maiden/ToySnacks snork-maiden/snacksbench \
--config https://huggingface.co/datasets/snork-maiden/ToySnacks/raw/main/config.yml \
--num-rows 100
```

I get this error about overlapping columns and I don't really understand what's the problem. I just want to create a larger dataset.
Could someone please explain how to solve this? Thanks in advance.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with README.md and scripts/extend_dataset/with_inference_client.py, then inspect the generation config at the linked config.yml alongside the source and destination dataset arguments. Reproduce the command with the provided ToySnacks inputs and determine how the overlapping-column error is triggered; done means the documented command can create the requested larger dataset without the ValueError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.