3
$\begingroup$

This question relates to the steps one would need to take in order to reproduce an answer from the DoWhy tutorial, using the EconML library code for heterogeneous causal effects.

In DoWhy, there is the following tutorial example to calculate the ATE (average treatment effect) of the Lalonde dataset:

https://microsoft.github.io/dowhy/dowhy_lalonde_example.html

In particular I am concerned with answer:

Causal Estimate is 1634.98683597

This was achieved in DoWhy based on propensity score weighting.

Now, EconML is a library based on Heterogeneous treatment effects, and in particular they advocate for "Double Machine Learning" models of the sort:

\begin{equation} \begin{split}Y =~& \theta(X) \cdot T + g(X, W) + \epsilon ~~~&~~~ \mathbb{E}[\epsilon | X, W] = 0 \\ T =~& f(X, W) + \eta & \mathbb{E}[\eta \mid X, W] = 0 \\ ~& \mathbb{E}[\eta \cdot \epsilon | X, W] = 0\end{split} \end{equation}

where where some treatment(s)/intervention(s)/action(s) $T$ were chosen and some outcome(s) $Y$ were observed and all the variables $W$ that could have potentially gone into the choice of $T$, and simultaneously could have had a direct effect on the outcome $Y$ (aka controls or confounders) are also recorder in the dataset.

https://econml.azurewebsites.net/spec/estimation/dml.html

In this model the "causal estimate" is reflected in parmeter $\theta(X)$. Heterogeniety arises because this is a function of $X$ - our required observable characteristics.

Naturally one can transform a problem based on CATE (conditional average treatment effect - which considers heterogeneity), to ATE by the law of iterated expectations:

\begin{equation} \mathbb{E}[Y(1) - Y(0)] = \mathbb{E}\left[\mathbb{E}[Y(1) - Y(0) \mid X ]\right] \end{equation}

Therefore it seems one should be able to transform CATE estimates into ATE estimates (i.e. use the EconML library to reproduce answers from DoWhy). However I'm not sure how to get an answer of $\theta(X) = 1634.986$ in this situation.

I've considered the following:

  • Evidently $\theta(X)$ is a function of $X$, but ATE estimates are not. Therefore can I input something like X = numpy.ones(N), to force the $\theta$ estimate to behave constant wrt $X$?

  • Perhaps I can try to remove the heterogeniety by off-loading $X$ into set $W$, and then making the left over $X$ in $\theta(X)$ to be random noise? i.e. X = numpy.random.randn(N)?

Question:

Essentially I can't see a straightforward / intuitive way to transform the causal estimate from Double Machine Learning (using EconML) to the equivalent ATE estimate - where I use this Lalonde causal estimate of ATE = 1634.986 as the example.

Does anyone have any idea how this problem should be approached?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Without focusing on EconML itself:

Averaging the HTE estimates should give us the ATE. This would rely on HTE estimation quality of course, but if HTEs are well-estimated (e.g. no over- or under-estimates effects for certain groups), the mean should be a reliable estimate of ATE. And obviously, if our sample is not representative of the population, averaging the HTEs may not generalize well to the true ATE. (It would be more reasonable to have some inverse probability weighting (IPW) to correctly estimate the ATE in that case.)

And now, for a quick example using dowhy and econml applying this rationale (Documentation around the use of DML can be found in: econml.dml.DML):

import numpy as np from sklearn.ensemble import RandomForestRegressor from sklearn.linear_model import LassoCV from econml.dml import DML import dowhy.datasets lalonde = dowhy.datasets.lalonde_dataset() np.random.seed(13) # Based on: # https://www.pywhy.org/dowhy/v0.11.1/example_notebooks/dowhy_lalonde_example.html from dowhy import CausalModel model=CausalModel(data = lalonde, treatment='treat', outcome='re78', common_causes='nodegr+black+hisp+age+educ+married'.split('+')) identified_estimand = model.identify_effect(proceed_when_unidentifiable=True) estimate = model.estimate_effect(identified_estimand, method_name="backdoor.propensity_score_weighting", target_units="ate", method_params={"weighting_scheme":"ips_weight"}) print("Causal Estimate is " + str(estimate.value)) # Causal Estimate is 1639.8401094783803 # Now we use EconML to estimate the same thing rf_reg = RandomForestRegressor(n_estimators=500, min_samples_leaf=2, random_state=13) lasso = LassoCV(cv=10, fit_intercept=False, random_state=13) est = DML(model_final = lasso, model_y=rf_reg, model_t=rf_reg, random_state=13) var_names = ["nodegr","black","hisp","age", "educ", "married"] # Fit the model est.fit(lalonde["re78"], lalonde["treat"], X=lalonde[var_names]) # Should have used a separate test set to evaluate this, # but for simplicity we use the same set # This is the ATE estimate est.ate(X=lalonde[var_names]) # This is the ATE estimate # 1620.70213037763 # As X was not `None` when fitting, we have to use it when making estimates. te_pred_rf_regressor = est.effect(lalonde[var_names]) # Calculate the HTEs te_pred_rf_regressor.mean() # This is the ATE estimate via averaging the HTEs # 1620.70213037763 from watermark import watermark print(watermark(packages="numpy,dowhy,sklearn,econml")) #numpy : 1.26.4 #dowhy : 0.11.1 #sklearn: 1.4.2 #econml : 0.15.0

As we can see the estimates in this case are fairly similar, 1639.8 and 1620.7 from the propensity score weighting and Double ML estimators respectively.

Improve this answer
$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.