When running crosstab, 'ttest_T' and 'ttest_p' will disappear if that feature has the same value across both positives and negatives
- Dominant language
- Jupyter Notebook
- Stars
- 201
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
https://github.com/dssg/triage/blob/59ffbe1f860e49eb175b4604a549c23cfb1c3fa5/src/triage/component/postmodeling/crosstabs/crosstabs.py#L164
If the feature has the same value across both positive and negative samples, the resulting rows from t-Test `ttest_T` and `ttest_p` will be dropped. We might want to keep those rows for consistency even if they are NaN by toggling the `dropna` argument to False in `results.stack()` function
```
results = results.stack(dropna=False)
```
Also
https://github.com/dssg/triage/blob/59ffbe1f860e49eb175b4604a549c23cfb1c3fa5/src/triage/component/postmodeling/crosstabs/crosstabs.py#L154-L156
This will always be `False` and will never print those. Despite that not sure why we need to print the shape.
Contributor guide
Research direction
Start by reading src/triage/component/postmodeling/crosstabs/crosstabs.py around lines 154-164, especially how the crosstab results are stacked and how the shape is printed. Check the behavior when a feature has one value across positive and negative samples; done means the ttest_T and ttest_p rows remain present, including NaN values where appropriate, and the shape output behavior is clarified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100