deepchecks / deepchecks/deepchecks
[DEE-3] [FEAT] Support additional file types in classification_dataset_from_directory
- Dominant language
- Python
- Stars
- 4.1k
- Forks
- 303
- PR merge metrics
- No merged PRs in 30d
Description
**Is your feature request related to a problem? Please describe.**
I am building a tool that aims to provide some information on image data. I would like to use the Vision Checks (Image property outliers, label property outliers, property label correlation), but I have come across a problem that not all (common) file types are accepted and need quite a lot of processing in order for the checks to work.
In the example Image Data Validation in 5 Minutes, JPG is used, but they do not work for other file/data types like .ARFF and especially .GZ (format where MNIST / CIFAR datasets often come in) before converting it to numpy arrays / pytorch tensors and splitting the data and images. As this process is quite tedious, an automated feature would be very helpful in this.
**Describe the solution you'd like**
I am looking for a method of getting the vision checks to work for file types like .ARFF and .GZ (format where MNIST / CIFAR datasets often come in), or some module that converts these file types to a numpy array or a pytorch tensor and then separates the images and the data automatically, as this process is quite tedious.
**Describe alternatives you've considered**
Converting the datasets to numpy arrays manually.
**Additional context**
Some examples of data formats:



[DEE-3](https://linear.app/deepchecks/issue/DEE-3/[feat]-support-additional-file-types-in-classification-dataset-from)
Contributor guide
Assessment
This issue has not been assessed yet.