huggingface / huggingface/dataset-viewer
Croissant for lmms-lab/LMMs-Eval-Lite is incorrect
- Dominant language
- Python
- Stars
- 899
- Forks
- 130
- Avg merge
- 9h 26m
- Merged PRs (30d)
- 14
Description
The [croissant](https://huggingface.co/api/datasets/lmms-lab/LMMs-Eval-Lite/croissant) for the [lmms-lab/LMMs-Eval-Lite dataset](https://huggingface.co/datasets/lmms-lab/LMMs-Eval-Lite) is not correct.
And more specifically in the "gqa/semantic/dependencies" field:
```
{
"@type": "cr:Field",
"@id": "gqa/semantic/dependencies",
"name": "gqa/semantic/dependencies",
"description": "Column 'semantic' from the Hugging Face parquet file.",
"dataType": "sc:Integer",
"source": {
"fileSet": {
"@id": "parquet-files-for-config-gqa"
},
"extract": {
"column": "semantic"
}
// <==== Here the transform is missing!
}
```
I don't know why `"transform": { "jsonPath": "dependencies" }` is missing here, given that it has been correctly added to all other subfields.
For comparison, the croissant on mlcroissant repo for the same dataset, with the correct `gqa/semantic/dependencies` subfield: https://github.com/mlcommons/croissant/blob/0b5cdfdcea72025fa4f7eaf8384e92eda291a118/datasets/1.0/huggingface-lmms-eval-lite/metadata.json
Can be loaded both with and without beam without any issue.
Contributor guide
Research direction
Compare the generated Croissant for lmms-lab/LMMs-Eval-Lite with the reference metadata in the linked mlcommons/croissant file, focusing on gqa/semantic/dependencies. Trace the dataset-viewer entry point that generates this field and verify that the resulting Croissant loads with and without Beam and includes the expected transform.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface
- Domain
- data
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100