dataset_list_files is missing files
- Dominant language
- Python
- Stars
- 7.5k
- Forks
- 1.4k
- Avg merge
- 3d 15h
- Merged PRs (30d)
- 11
Description
Greetings -
2 issues:
1) Using the api client, taking the [Meta Kaggle](https://www.kaggle.com/kaggle/meta-kaggle) dataset as an example, it does not seem to return all of the files under the dataset. 2) Also, the dataset_list_files method fails. Please let me know if I am missing something. Thank you.
```
metakaggle_ds = kaggle.api.dataset_list(search="Meta Kaggle", user="kaggle")[0]
vars(metakaggle_ds)
# kaggle.api.dataset_list_files(metakaggle_ds.ref) # fails with TypeError: __repr__ returned non-string (type NoneType)
# manually:
dataset = metakaggle_ds.ref
if '/' in dataset:
kaggle.api.validate_dataset_string(dataset)
dataset_urls = dataset.split('/')
owner_slug = dataset_urls[0]
dataset_slug = dataset_urls[1]
else:
owner_slug = kaggle.api.get_config_value(kaggle.api.CONFIG_NAME_USER)
dataset_slug = dataset
dataset_list_files_result = kaggle.api.process_response(
kaggle.api.datasets_list_files_with_http_info(owner_slug=owner_slug,
dataset_slug=dataset_slug))
ds_files = [x['ref'] for x in dataset_list_files_result['datasetFiles']]
ds_files # returns only some of the files in the dataset, e.g. Users.csv and Teams.csv are missing, and they also cannot be downloaded either via kaggle.api.dataset_download_file()
```
Contributor guide
Assessment
This issue has not been assessed yet.