ml-explore / ml-explore/mlx-examples
load_custom_hf_dataset not handling the text_feature argument properly
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 9k
- Forks
- 1.2k
- PR merge metrics
- No merged PRs in 30d
Description
If you use a hf_dataset configuration such as (for e.x.)
hf_dataset:
name: "Open-Orca/OpenOrca"
train_split: "train[:90%]"
valid_split: "train[-10%:]"
text_feature: "response"
It is supposed to work the same as the (local) text data format, but it comes up against:
if prompt_feature and completion_feature:
return CompletionsDataset(ds, tokenizer, prompt_feature, completion_feature)
elif text_feature:
return Dataset(train_ds, text_key=text_feature)
else:
raise ValueError(
"Specify either a prompt and completion feature or a text "
"feature for the Hugging Face dataset."
)
which errors out because train_ds is not defined
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Search for load_custom_hf_dataset and inspect the branch shown in the issue, especially the text_feature path that references train_ds. Compare it with the local text data format and verify the Hugging Face configuration using Open-Orca/OpenOrca; done means text_feature loads the selected dataset without the undefined-variable error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python
- Domain
- data, machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100