statsmodels / statsmodels/statsmodels

Design: register a transform in fit for results

Open
#2,373 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

comp-base design type-enh
Dominant language
Python
Stars
11.6k
Forks
3.6k
Avg merge
7h 37m
Merged PRs (30d)
96

Description

Just an idea, I don't know yet whether this is feasible.

use case:
In penalized estimation, the penalization is often calibrated or predefined for standardized coefficients, standardized data (rescaled exog).

If the user uses a transformation prior to creating the model, then it might still be of interest to have the results, tests and reports for the original scale. That is we fit in one parameterization or scaling and have the Results using a different one.

This is a generalization of fit_constrained and fit_transformed that allows user to use only the second half of the transformation.

Problem with this:

While it is easy to transform params and cov_params, there are many results that directly depend on the attached exog, like fittedvalues, resid, rsquared and similar. The attached exog would have the wrong scaling, if the user rescaled it.

fit_constrained and fit_transformed use an auxiliary model that contains the transformed data, but still has the original data in the main model.

(Doesn't sound good anymore.)

correction

If only exog is transformed but not endog, then fittedvalues and resid would still be independent of linear or affine transformation of exog.
Predict, Margins and others would depend on the original scaling.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No file, test, or entry point is named. Start by reviewing the existing fit_constrained and fit_transformed implementations, then trace which results depend on exog scaling. Done would require an agreed design and tests covering transformed parameters, covariance, prediction, margins, fitted values, residuals, and reports.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.