neurostuff / neurostuff/PyMARE

Clarify or check Estimator input shapes

Open
#37 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

documentation enhancement
Dominant language
Python
Stars
58
Forks
16
Avg merge
5h 7m
Merged PRs (30d)
4

Description

I was trying to run one of the Estimators using weighted_least_squares without initializing a Dataset and was getting confusing errors from numpy.einsum before I realized that inputs need to be 2D no matter what. We can coerce 1D inputs to 2D with a new Estimator._validate_inputs() method, or we can just update the docstrings to clarify requirements.

BTW, based on variable convention, I think it's reasonable to assume that X must be 2D, but it's not obvious that y, v, etc. should be 2D as well, and indeed, there's an obvious error about shape if X is 1D, but no shape check for y, v, etc.

To replicate:

y = np.random.random(10)
v = np.random.random(10) ** 2
X = np.random.random((10, 1))
est = pymare.estimators.DerSimonianLaird()
est.fit(y=y, v=v, X=X)
print(est.results.to_df())

Results in:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-4-41d64261b276> in <module>()
      3 X = np.random.random((10, 1))
      4 est = pymare.estimators.DerSimonianLaird()
----> 5 est.fit(y=y, v=v, X=X)
      6 print(est.results.to_df())

~/Documents/tsalo/PyMARE/pymare/estimators/estimators.py in fit(self, dataset, **kwargs)
     78                     kwargs[name] = getattr(dataset, name)
     79 
---> 80         self.params_ = self._fit(**kwargs)
     81         self.dataset_ = dataset
     82 

~/Documents/tsalo/PyMARE/pymare/estimators/estimators.py in _fit(self, y, v, X)
    180 
    181         # Estimate initial betas with WLS, assuming tau^2=0
--> 182         beta_wls, inv_cov = weighted_least_squares(y, v, X, return_cov=True)
    183 
    184         # Cochrane's Q

~/Documents/tsalo/PyMARE/pymare/stats.py in weighted_least_squares(y, v, X, tau2, return_cov)
     24 
     25     # Einsum indices: k = studies, p = predictors, i = parallel iterates
---> 26     wX = np.einsum('kp,ki->ipk', X, w)
     27     cov = wX.dot(X)
     28 

<__array_function__ internals> in einsum(*args, **kwargs)

~/anaconda/envs/python3/lib/python3.6/site-packages/numpy/core/einsumfunc.py in einsum(*operands, **kwargs)
   1354     # If no optimization, run pure einsum
   1355     if optimize_arg is False:
-> 1356         return c_einsum(*operands, **kwargs)
   1357 
   1358     valid_einsum_kwargs = ['out', 'dtype', 'order', 'casting']

ValueError: einstein sum subscripts string contains too many subscripts for operand 1

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the example and start in pymare/estimators/estimators.py, following fit into the estimator implementation and pymare/stats.py's weighted_least_squares. Determine the intended handling of 1D y and v versus X, then ensure the chosen validation or documentation makes those requirements explicit and avoids the confusing einsum failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
numpy, python
Domain
data
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.