deepchem / deepchem/deepchem

DiskDataset assumes the X field is populated, in several places

Open
#4,060 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
7k
Forks
2.3k
PR merge metrics
No merged PRs in 30d

Description

While it is possible to easily create a DiskDataset that does not have the X field populated, there are lots of places in the DiskDataset code that assume that it is.

There is a strong use case for being able to make DiskDataset objects with, say, just a y field. For doing inference on large datasets, it makes sense to create a DiskDataset to contain the resulting generated data, and the most reasonable place to put this is in the y field.

There are several places in the code this assumption is made; for example in get_shard() it's assumed that the X file is there. Notably, this is not checked for on DiskDataset creation.

Contributor guide

Open the contributing guide

Research direction

Begin at DiskDataset creation and get_shard(), then locate the other DiskDataset paths that assume an X file exists. Done means a DiskDataset containing only y can be created and used through those paths without requiring X, with regression coverage for the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.