py-why / py-why/EconML

Multiple Treatments with Econml

Open
#930 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
4.8k
Forks
827
PR merge metrics
No merged PRs in 30d

Description

Hi,
I greatly enjoy the EconML library. However, regarding multiple treatments, there is an issue I could not figure out. I would really appreciate your help.
Here is the brief of my problem:

I have 2 binary columns (email_campaign,social_media_ad) with an X variable and binary outcome.I ran a combined treatment with CausalForestDML and ran separate CausalForestDML separately for each treatment. why I get different ate results? When running multiple treatments, when I set T0=0,T1=1 why the ate result is different than running a separate model with only treatment email_campaign? The combined treatment column is 0 when email_campaign and social_media_ad is zero, 1 when social_media_ad is 1 and social_media_ad is 0 , 2 when email_campaign is 1 and social_media_ad is 0, 3 when both are 1. A sample of the data is:

image

import pandas as pd
import numpy as np
from econml.dml import CausalForestDML
from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor
np.random.seed(123)

Sample data (replace with your actual data)

data = pd.DataFrame({
'Customer ID': range(1, 1001),
'Sales': np.random.randint(0, 1000, 1000),
'churn': np.random.randint(0, 2, 1000),
'Email Campaign': np.random.randint(0, 2, 1000),
'Social Media Ad': np.random.randint(0, 2, 1000)
})

Create the combined treatment variable

data['Combined Treatment'] = data['Email Campaign'] * 2 + data['Social Media Ad']
data.columns=data.columns.str.lower().str.replace(' ','_')

Define features and target variable

X = data[['sales']]
T = data['combined_treatment']
Y = data['churn']

Initialize the CausalForestDML model

est = CausalForestDML(
model_t=RandomForestClassifier(random_state=123),
model_y=RandomForestRegressor(random_state=123),
discrete_treatment=True,random_state=123
)

Fit the model

model_est=est.fit(Y, T, X=X)

The ate result of each treatment:
est.ate(X,T0=0,T1=1) --> -0.0016 Social_media_ad ( combined_treatment==1)
est.ate(X,T0=0,T1=2) --> -0.033 The email_campaign (combined_treatment==2)
est.ate(X,T0=1,T1=2) --> -0.032
est.ate(X,T0=0,T1=3) --> -0.051

Email:
est_mail = CausalForestDML(
model_t=RandomForestClassifier(random_state=123),
model_y=RandomForestRegressor(random_state=123),
discrete_treatment=True,random_state=123
)

        est_mail.fit(Y, data["email_campaign"], X=X)
        est_mail.ate(X)  --> -0.019

In the above example, T0=0,T1=2 means the treatment of email_campaign. My question is why it yields different results with multiple treatments and separate treatments? How to utilize the multiple treatments approach in EconML?
Social media ad:
est_social_media_ad = CausalForestDML(
model_t=RandomForestClassifier(random_state=123),
model_y=RandomForestRegressor(random_state=123),
discrete_treatment=True,random_state=123
)

      est_social_media_ad .fit(Y, data["social_media_ad"], X=X)
      est_social_media_ad .ate(X) -->0.010

In the above example, T0=0,T1=1 means the treatment of social_media_ad. The result from multiple treatment model is negative but in the single treatment model is positive. Why?

Note:
1- I receive even contrasting (negative vs positive) results when running on different datasets.
2- I receive inconsistent results even if two treatment variables are totally independent, meaning when each customer receives only one treatment.

Best

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the supplied CausalForestDML examples and reproduce the differing ATE results for combined and separate treatments. Read the CausalForestDML treatment handling and ate(T0, T1) entry points, then compare the treatment encoding and fitted models. Done means explaining whether the contrasting estimates are expected or identifying a reproducible implementation defect.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.