statsmodels / statsmodels/statsmodels

t_test `can't test a constant constraint`

Open
#1,704 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

comp-base
Dominant language
Python
Stars
11.6k
Forks
3.6k
Avg merge
7h 37m
Merged PRs (30d)
96

Description

I'm trying to use t_test to get the covariance for a degenerate linear transformation, but patsy complains.

question: Should this be allowed?

Note: for a while now I have been using t_test to get the results for linear transformations and not just for "t-tests"

    tt = res3.t_test(transf3.transf_mat, transf3.constant.squeeze())
  File "e:\josef\eclipsegworkspace\statsmodels-git\statsmodels-all-new2_py27\statsmodels\statsmodels\base\model.py", line 1050, in t_test
    LC = DesignInfo(self.model.exog_names).linear_constraint(r_matrix)
  File "build\bdist.win32\egg\patsy\design_info.py", line 220, in linear_constraint
  File "build\bdist.win32\egg\patsy\constraint.py", line 392, in linear_constraint
  File "build\bdist.win32\egg\patsy\constraint.py", line 58, in __init__
ValueError: can't test a constant constraint

>>> transf3.transf_mat
array([[ 0.,  1.,  0.],
       [ 0.,  0.,  1.],
       [ 1.,  0.,  0.],
       [-0., -0., -0.],
       [-0., -0., -0.]])
>>> transf3.constant.squeeze()
array([ 0.,  0.,  0.,  0.,  1.])
>>> res3.model.exog.shape
(20, 3)

example is for constraint estimation, the covariance matrix in the constraint full model (parameterized for original exog) is degenerate or rank reduced.

>>> cov_params3 = transf3.transf_mat.dot(res3.cov_params()).dot(transf3.transf_mat.T)
>>> np.sqrt(np.diag(cov_params3))
array([ 0.59285072,  0.31334777,  0.30669526,  0.        ,  0.        ])

>>> print res3_ols.bse
[ 0.59285072  0.31334777  0.30669526]

(example based on "dirty" development script for estimation with linear constraints)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in statsmodels/base/model.py at Results.t_test, then follow its call to patsy's DesignInfo.linear_constraint. Reproduce the shown rank-reduced constraint with the supplied matrices and determine whether constant constraints should be accepted; done means the intended covariance and constraint results work without patsy rejecting the input.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.