huggingface / huggingface/aisheets
ValueError: Columns defined in both source dataset and generation config
- Dominant language
- TypeScript
- Stars
- 1.6k
- Forks
- 140
- Avg merge
- 1m
- Merged PRs (30d)
- 1
Description
I am trying to create a large dataset with the script provided in the README.md. Here is my script:
```python
hf jobs uv run --namespace $NAMESPACE \
-s HF_TOKEN=$HF_TOKEN \
https://github.com/huggingface/aisheets/raw/refs/heads/main/scripts/extend_dataset/with_inference_client.py \
snork-maiden/ToySnacks snork-maiden/snacksbench \
--config https://huggingface.co/datasets/snork-maiden/ToySnacks/raw/main/config.yml \
--num-rows 100
```
I get this error about overlapping columns and I don't really understand what's the problem. I just want to create a larger dataset.
Could someone please explain how to solve this? Thanks in advance.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with README.md and scripts/extend_dataset/with_inference_client.py, then inspect the generation config at the linked config.yml alongside the source and destination dataset arguments. Reproduce the command with the provided ToySnacks inputs and determine how the overlapping-column error is triggered; done means the documented command can create the requested larger dataset without the ValueError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100