deepchecks / deepchecks/deepchecks

[DEE-3] [FEAT] Support additional file types in classification_dataset_from_directory

Open
#2,199 1 comment 0 reactions 0 assignees View on GitHub
feature vision
Dominant language
Python
Stars
4.1k
Forks
303
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**
I am building a tool that aims to provide some information on image data. I would like to use the Vision Checks (Image property outliers, label property outliers, property label correlation), but I have come across a problem that not all (common) file types are accepted and need quite a lot of processing in order for the checks to work.
In the example Image Data Validation in 5 Minutes, JPG is used, but they do not work for other file/data types like .ARFF and especially .GZ (format where MNIST / CIFAR datasets often come in) before converting it to numpy arrays / pytorch tensors and splitting the data and images. As this process is quite tedious, an automated feature would be very helpful in this.

**Describe the solution you'd like**
I am looking for a method of getting the vision checks to work for file types like .ARFF and .GZ (format where MNIST / CIFAR datasets often come in), or some module that converts these file types to a numpy array or a pytorch tensor and then separates the images and the data automatically, as this process is quite tedious.

**Describe alternatives you've considered**
Converting the datasets to numpy arrays manually.

**Additional context**
Some examples of data formats:
![image](https://user-images.githubusercontent.com/74533268/207837360-be09c9ad-93a7-4997-bff9-3bcd460e2059.png)
![image](https://user-images.githubusercontent.com/74533268/207837455-eeefa8c0-0aed-41db-bafe-1e3eff265c70.png)
![image](https://user-images.githubusercontent.com/74533268/207837509-bb0adbbf-020d-4e2f-8d75-5afab703463f.png)

[DEE-3](https://linear.app/deepchecks/issue/DEE-3/[feat]-support-additional-file-types-in-classification-dataset-from)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.