pydata / pydata/patsy

Standardize transform should handle edge case where variable has zero variance

Open
#85 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
990
Forks
106
Avg merge
7d 34m
Merged PRs (30d)
1

Description

In edge cases where a variable has zero variance, standardize should do the same as center rather than producing an error.

Suppose I am fitting a linear model with a large number of random covariates. I'd like to standardize them and fit a regularized model (e.g. sklearn.linear_model.Ridge). If there's some probability that a given covariate has zero variance, I'd like my code to run (and give that covariate a coefficient of zero) rather than crash.

Toy example:

import patsy
data = {"x1": [1, 2, 3, 4], "x2":[5, 5, 5, 5]}
mat = patsy.dmatrix("standardize(x1) + center(x2)", data)  # Works
mat = patsy.dmatrix("standardize(x1) + standardize(x2)", data)  # Fails! Should handle this edge case

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the implementation and tests for patsy’s standardize and center transforms, then reproduce the issue with the provided patsy.dmatrix example. Done means standardize accepts a zero-variance variable without raising an error and produces the centered result, while existing non-constant behavior remains unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.