huggingface / huggingface/dataset-viewer
The parameters of an endpoint should not change the response format
- Dominant language
- Python
- Stars
- 899
- Forks
- 130
- Avg merge
- 9h 26m
- Merged PRs (30d)
- 14
Description
The optional parameters should only change the response's content, not structure.
For example, the `length` parameter in /rows reduces the number of returned rows.
But for /parquet, for example, if we ask for the config level (https://datasets-server.huggingface.co/parquet?dataset=mnist), we get the list of features along with the list of files, while we don't have features when we only ask for the dataset level (https://datasets-server.huggingface.co/parquet?dataset=mnist&config=mnist). Also, for /info, the structure of `dataset_info` is not the same for dataset level and config level.
For /size, the fields' names and types change depending on whether the config parameter is passed or not. For example, https://datasets-server.huggingface.co/size?dataset=mnist gives `.size.configs`, while https://datasets-server.huggingface.co/size?dataset=mnist&config=mnist give `.size.config`.
Similarly, the `failed` and `pending` entries are weird. They only show for "aggregated" levels (i.e., dataset if the response is generated at config level, dataset, and config if the response is generated at split level). Currently:
- /splits, dataset level
- /parquet, dataset level
- /info, dataset level
- /size, dataset level
- /opt-in-out-urls, dataset and config levels
About "failed" and "pending", also note that their type is different depending on the endpoint. Just one example: "failed" in /splits return the error, while "failed" in /parquet return the parameters of the previous job.
Also: in parquet and info, instead of not setting "split", we set it to None (which gives `null` in JSON, instead of not having the field).
Contributor guide
Research direction
Start by comparing the documented responses for /rows, /parquet, /info, /size, /splits, and /opt-in-out-urls at their different levels and parameter combinations. Define the consistent fields and types required for each endpoint, including failed, pending, and split, then verify that the responses no longer change structure when optional parameters are used.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100