Is this the expected behavior of patsy when building a design matrix of a two-level categorical variable without an intercept?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 990
- Forks
- 106
- Avg merge
- 7d 34m
- Merged PRs (30d)
- 1
Description
cloned from stackoverflow: http://stackoverflow.com/questions/35944841/is-this-the-expected-behavior-of-patsy-when-building-a-design-matrix-of-a-two-le
(patsy v0.4.1, python 3.5.0)
I would like to use patsy (ideally through statsmodels) to build a design matrix for regression.
The patsy-style formula that I would like to fit is
response ~ 0 + category
where category is a two-level categorical variable. The 0 + ... is supposed to indicate that I do not want the implicit intercept term.
The design matrix that I expect has a single column with zeros and ones indicating whether category has the base-level (0) or the other level (1).
The following code:
import pandas as pd
import patsy
df = pd.DataFrame({'category': ['A', 'B'] * 3})
patsy.dmatrix('0 + category', data=df)
Outputs:
DesignMatrix with shape (6, 2)
category[A] category[B]
1 0
0 1
1 0
0 1
1 0
0 1
Terms:
'category' (columns 0:2)
which is singular and not what I want.
When I instead run
import pandas as pd
import patsy
df = pd.DataFrame({'category': ['A', 'B'] * 3})
patsy.dmatrix('category', data=df)
the output is
DesignMatrix with shape (6, 2)
Intercept category[T.B]
1 0
1 1
1 0
1 1
1 0
1 1
Terms:
'Intercept' (column 0)
'category' (column 1)
which is correct for the model which includes an intercept, but still not what I want.
Is the output without an intercept the intended behavior? If so, why?
Am I just confused about how this design matrix is supposed to work with standard coding?
I know that I can edit the design matrix to make my regression work the way I intend, but if this is a bug I'd like to see it fixed in patsy.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the provided Python and patsy.dmatrix reproducer with patsy 0.4.1, then compare the two displayed design matrices and read the categorical encoding behavior it exercises. Done means determining whether the two-column no-intercept result is intended and documenting or correcting that behavior with a clear regression check.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100