py-why / py-why/EconML

Divide by zero encountered in true_divide invalid value encountered in matmul

Open
#455 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
4.8k
Forks
827
PR merge metrics
No merged PRs in 30d

Description

Hi all,

I am dealing with a panel estimation and am fitting the following model:

est = SparseLinearDML(featurizer=PolynomialFeatures(degree=4, include_bias=False),model_y= LassoCV(cv=[(fold00, fold11), (fold11, fold00)], tol=0.0001,n_alphas = 1000, fit_intercept=False,max_iter=100000), 
                                       model_t= MultiTaskElasticNetCV(cv=[(fold00, fold11), (fold11, fold00)],n_alphas = 1000, fit_intercept=False,max_iter=100000),
                                       cv = [(fold0, fold1), (fold1, fold0)],
                                       #linear_first_stages=True,
                                       fit_cate_intercept=False)

However, the error in the title appears. I know it is something about the presence of Nan values or inf but my data look good actually (see attached file data.csv). In particular, I am using the following variables: ln_stdunits ln_price avg_price_atc sales_firms, avg_price_atc, generics, quarter_formatted as pd.datetime() variable, num_prd to equally split the variables for each Productlocal within a molecule.

EDIT: the X is as you suggested for panel data estimation, i.e. a mix individual and time dummies:

for g in np.arange(n_groups):
            for i in np.arange(n_products_per_group):
                for j in np.arange(n_months):
                    if n_products_per_group>1:
                        index_i = n_products_per_group*g + i
                        minus_i = (np.arange(n_products) >= (n_products_per_group*g))
                        minus_i &= (np.arange(n_products) < (n_products_per_group*(g+1)))
                        minus_i &= (np.arange(n_products) != (index_i))
                        p_minus[index_i, j] = np.mean(p[minus_i, j])
                    
        for g in np.arange(n_groups):
            for i in np.arange(n_products_per_group):
                for j in np.arange(n_months):
                    index_i = n_products_per_group*g + i
                    #INDICE INDIVIDUO PER TUTTI GLI INDIVIDUI COMPRESO IL PRIMO
                    #INFATTI IL PRIMO ERA PRESO DI BASELINE SOLO IN CASO DI COLLINEARITA (I.E. SE USO LINEARDML)
                    X[index_i, j, :(n_products)] = 1.0 * (np.arange(0, n_products) == (index_i))
                    #INDICE GRUPPO TRANNE PRIMO GRUPPO
                    X[index_i, j, (n_products):(n_products + n_groups - 1)] =  1.0 * (np.arange(1, n_groups) == g)
                    #INDICE QUARTER TRANNE PRIMO QUARTER
                    X[index_i, j, (n_products + n_groups - 1):] = 1.0 * (np.arange(1, n_months) == j)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the SparseLinearDML setup with the LassoCV and MultiTaskElasticNetCV models shown, using the attached data.csv and the constructed X panel features. Trace the divide-by-zero and invalid matmul warnings through the fitting path and verify whether the reported inputs contain problematic values. Done means identifying a reproducible failure cause and documenting or testing the expected behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, scikit-learn
Domain
data, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.