py-why / py-why/EconML

DML: Should the treatment effect be constant at a given X, over various values of the treatment?

Open
#475 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
4.8k
Forks
827
PR merge metrics
No merged PRs in 30d

Description

Hello.

I am running the example here: https://github.com/microsoft/EconML/blob/master/notebooks/CustomerScenarios/Case%20Study%20-%20Customer%20Segmentation%20at%20An%20Online%20Media%20Company.ipynb

I am confused by what is being estimated I think. After this

est = CausalForestDML(
    model_y=GradientBoostingRegressor(), model_t=GradientBoostingRegressor()
)
est.fit(log_Y, log_T, X=X, W=W, inference="blb")

I expected that since this was a non-parametric estimator, compared to linear DML that the treatment effect should differ at various values for T given a constant X.

When I run these, which I think are producing the CATE as the treatment goes from T0 to T1 at the given X values for each observation, they are all the same. These are all 1 unit apart for T0 and T1.

est.effect(X_test, T0=-0.22, T1=.78)
est.effect(X_test, T0=1.22, T1=2.22)

Further, if I produce what I think is the slope of the CATE at a given value of a treatment, for a fixed X, they are all the same:

est.marginal_effect(T= -0.22, X = X_test)
est.marginal_effect(T= 1, X = X_test)

Likewise if I produce the ATE for a given X, which I think are simply the mean of the above (est.effect(X_test, T0=0, T1=1)) for different values of T0 and T1, the results are all the same, The average marginal is the same too regardless of the value of T it is evaluated at....

est.ate(T0= 0 , T1 =1 ,X = X_test)
est.ate(T0= -1 , T1 =0 ,X = X_test)

est.marginal_ate(T= -0.22, X = X_test)
est.marginal_ate(T= 0, X = X_test)

Are my understandings incorrect that these should differ?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the Customer Segmentation notebook and the CausalForestDML calls shown in the issue; compare the semantics of effect, marginal_effect, ate, and marginal_ate at different treatment values. Done means documenting whether the observed constant results are expected and clarifying the example or API guidance accordingly.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python
Domain
data, machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.