py-why / py-why/EconML

DynamicDML() issue: AttributeError: Provided crossfit folds contain training splits that don't contain all treatments DynamicDML

Open
#859 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
4.8k
Forks
827
PR merge metrics
No merged PRs in 30d

Description

Hello,

When calling DynamicDML() as such:

est = DynamicDML(model_y = model_y, model_t=model_t, discrete_treatment=True)
est.fit(Y, T, X=X, W=None, groups=groups)

Here Y, T , X and groups are in long format and have the following shapes:

((32382,), (32382,), (32382, 8)), (32382,)

where n=N*Time=32,382; N=1542 cross-sectional units and Time=21 months and groups has N distinct ids corresponding to the distinct cross-sectional units.

I have already balanced the panel. T is a binary and discrete treatment and I see the default value for discrete_treatment is False, when setting instantiating with discrete_treatment=True I get:

AttributeError: Provided crossfit folds contain training splits that don't contain all treatments

arising from

if np.any(np.all(Target == 0, axis=0)) or (not np.any(np.all(Target == 0, axis=1))):
     raise AttributeError("Provided crossfit folds contain training splits that " +
"don't contain all treatments")

and it appears Target is a 1-hot encoding of T; if so, then this condition: (not np.any(np.all(pd.get_dummies(T,dtype=int) == 0, axis=1))) is True leading to the Attribute error. The way I am coding T is for each cross-sectional unit & month observation T=0 if that unit is not treated yet and T=1 once they become treated and remains 1; while controls have T=0 for all months. I imagine this is fine?

I'm using RandomForestClassifier for model_t and GradientBoostingRegressor for model_y.

The correct instantiation would be the one with discrete_treatment=True so that is the error I am more concerned about, just providing full context.

I get the following error (with discrete treatment is False):
Co-variance matrix is underdetermined. Inference will be invalid!

this holds with or without the inclusion of X, which has been standardized such that features have zero mean and unit variance.

Thanks,

Saman

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the DynamicDML.fit call with the provided long-format shapes, grouped panel ids, and binary treatment. Inspect the cross-fitting validation around the shown Target condition and compare the discrete_treatment=True and False paths; done means the reported treatment-fold behavior and covariance warning have a clear, verified diagnosis or a corresponding regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, scikit-learn
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.