neurostuff / neurostuff/PyMARE

Handling of missing data in Estimators

Open
#42 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
58
Forks
16
Avg merge
5h 7m
Merged PRs (30d)
4

Description

Per @tyarkoni in https://github.com/neurostuff/PyMARE/issues/40#issuecomment-649137976:

I don't recommend treating voxels with missing data for studies as if the estimates and variances are 0. My naive expectation is that PyMARE estimators will always return a NaN value if you do that (if they don't, let me know!), which would be fine since you could just mask those voxels out. BUT even if that's true, I don't think you want to rely on PyMARE to do the right thing here. If you know you have missing data for at least one study at a voxel, I suggest either setting the estimates and variances for those studies to NaN or (better) just masking them out before you hand off to PyMARE in the first place.

Using DerSimonianLaird as an example estimator, here is the behavior I've found:

  • Any NaNs in y --> tau2 = NaN
  • All zeros in y --> tau2 = 0
  • Any NaNs in v --> tau2 = NaN
  • Any zeros in v --> tau2 = NaN

I think the behavior we want is to ignore any studies with NaNs in either y or v, and maybe raise an error when v is 0 (with the recommendation to fill missing data with NaNs instead of zeros) and raise a warning when y is 0. And of course, if all studies have NaN, then all parameters estimated by the estimator should be NaN.

It would then be the user's responsibility to ensure that missing data is represented with NaNs. On NiMARE's side, I can open an issue to do this automatically in MetaEstimator.fit(). I don't think masking is feasible for NiMARE, given that missing data may vary by both study and voxel, so we'd end up with varying numbers of studies contributing to each voxel's meta-analysis.

WDYT?

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the DerSimonianLaird estimator behavior and the MetaEstimator.fit() path mentioned in the issue. Check how NaNs and zero values in y and v are currently handled. Done should include ignoring studies with missing values, defining the requested zero-value warning or error behavior, and returning NaN when all studies are missing.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.