NeuroTechX / NeuroTechX/moabb

Data quality level

Open
#276 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

dataset enhancement moabb
Dominant language
Python
Stars
1.1k
Forks
264
Avg merge
1d 13m
Merged PRs (30d)
23

Description

We could add a data quality level either to BaseDataset or to Evaluation. The idea would be something like:

Level 0 (All data): Load everything that is defined in the dataset.
Level 1 (Sane data): Load data where the basic information corresponds to the description in the documentation/paper. E.g. only load subjects where Target/NonTarget ratios are correct and the number of events etc. is correct.
Level 2 (Hand-vetted data): Probably a lot of effort, but in this setting we could omit subjects / sessions where a large number of artifacts is produced e.g. due to eye blinks or faulty electrodes.
Level -1 (undefined): New dataset PRs need to be vetted, until that is done we could assign some arbitrary level to these.

Additionally, after a dataset has been checked, e.g. via a sanity check script, it could get a badge / tag on the documentation data checked and loading correctly and other datasets get either data not checked, use at your own risk or there are issues with this dataset, see Github #X

I am open to other suggestions / ideas.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the BaseDataset and Evaluation entry points mentioned in the issue, along with how dataset documentation currently represents validation status. Determine whether the project wants quality levels, documentation badges, or both. Done would require an agreed design, an implementation plan, and corresponding documentation for checked, unchecked, and known-problematic datasets.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.