statsmodels / statsmodels/statsmodels

REF: cleanup wald test contrast df_xxx

Open
#2,218 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

comp-base comp-stats type-cleanup type-refactor
Dominant language
Python
Stars
11.6k
Forks
3.6k
Avg merge
7h 37m
Merged PRs (30d)
96

Description

Bad naming !

ContrastResults uses df_denom for the p-value
self.pvalue = self.dist.sf(self.statistic, df_denom)

This should be a serious bug. The relevant lines show that they have test coverage.
(This got me worried for some time. test failure in statsmodels.base.tests.test_generic_methods.TestGeneric....test_ftest_pvalues if I make a change to df in chi2, no failure in regression or discrete tests, i.e. only conistency checks with verified pvalues)

However, df_denom in chi2 is not the same as df_denom in the F-test. It's the same as df_num.

https://github.com/statsmodels/statsmodels/commit/dbbf05a02d9490fe4c33d3ef0bf5c4fef9350db0#diff-b165b4bd4e10edd0dfb485e47c562b2cL1518

my problem: I never know which is the numerator and which the denominator
switch to generic df_constraints instead of df_num

aside: df_constraints for wald test are defined in terms of shape. Using rank should be safer. Also we don't check for "estimability" which I still haven't fully figured out.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with ContrastResults and the linked commit, then inspect the Wald-test degrees-of-freedom handling in the chi2 path. Run statsmodels.base.tests.test_generic_methods.TestGeneric....test_ftest_pvalues and the regression or discrete tests mentioned in the issue. Done means the degrees-of-freedom names consistently describe their use and the existing verified p-value checks still pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, testing
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.