mne-tools / mne-tools/mne-python
Reconsider how missing values are represented in auto-generated epochs metadata
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.5k
- Forks
- 1.6k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 100
Description
Originally posted by @cbrnr in https://github.com/mne-tools/mne-python/pull/12931#discussion_r1828854826
Pandas has many ways to represent missing data; this choice uses None, whereas other columns use NaN. Although pandas correctly treats all of these values as missing, we could take advantage of nullable extension data types, which add proper support for missing values, most notably to create various nullable integer types (Int8, Int16, ..., UInt8, UInt16, ...) and a string type.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the code that generates epochs metadata and review how its missing values are represented across columns. Compare the available pandas nullable extension dtypes and establish a consistent choice; done means the representation is selected and applied consistently, with affected behavior covered by the relevant tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100