When treatment is binary, the Causal Treatment personality in JMP enables you to choose between four estimation techniques: inverse probability weighting with ratio adjustment (IPWR), regression adjustment (REGADJ), augmented inverse probability weighting (AIPW), and propensity score matching. You may be comfortable with one of these estimation methods, or you may have no idea where to begin. Along with the Introduction to Causal Treatment blog post and the JMP 19 help documentation, this article should help you decide which results to report.
If you are interested in the effect of a binary treatment on an outcome, the Causal Treatment personality allows you to specify two models: a treatment model and a response model. The treatment model estimates the probability that an observation received treatment. The response model estimates the expected value of the outcome at both levels of the treatment. IPWR and propensity score matching use information about the treatment model to estimate causal effects. REGADJ uses information on the response model. AIPW uses information on both.
If you have read the Intro to Causal Treatment blog post, you might remember the three assumptions necessary for performing causal inference on observational data: positivity, consistency, and conditional exchangeability. For correct causal inferences, these assumptions must be made, and you must also assume you are modeling the conditional distributions of the treatment given the confounding set, or the response given the treatment and the confounding set, correctly. This is a strong assumption, since it’s likely that we don’t know much about the data-generating processes that produce the samples we have in front of us. However, since the AIPW estimator uses information from both the specified treatment and response models, it has a better chance of producing the correct answers to causal questions.
The AIPW estimator is known as a doubly robust estimator. While the IPWR, propensity score matching, and REGADJ estimators require that you get either the treatment or response model “right,” the AIPW estimator gives you two chances. The AIPW estimator will be consistent if either the treatment or response model is specified correctly (Scharfstein et al., 1999). Consistency means that as your sample size gets larger, you get close to the true causal effect you are hoping to estimate. Double robustness is a popular term in causal inference because it is a desirable property that allows you to use all the information you have to get as close as possible to the target estimand.
The Causal Treatment personality is great because it allows you to choose the estimator you want to use. If you are inclined to use one of the other three estimators, here are some things to keep in mind:
- The propensity score matching estimator can typically only give you an estimate of the average treatment effect on the treated (ATET) and not the average treatment effect (ATE) (Abadie & Imbens, 2006). Clearly define which causal estimand you are interested in before considering this method.
- Methods that use the treatment model (i.e., IPWR and propensity score matching) may work better in smaller samples, since the number of confounders could outgrow the number of observations, leading to unreliable results in the outcome regression (Benedetto et al., 2018). IPWR and matching collapse all confounder information into one number, the propensity score, which helps mitigate this issue.
- REGADJ may have an advantage over IPWR and propensity score matching when propensity scores are extreme. Many very small or very large propensity scores could lead to the exclusion of many observations from the analysis, decreasing the sample size and, therefore, decreasing the precision of the estimate (Elze et al, 2017).
It is not recommended to compare the results of all four estimators post-hoc. Hopefully, this post can help you choose the proper estimator for your data a priori, while the other estimation methods can be used for sensitivity analyses.
On top of these four estimation methods, many other estimation methods can be implemented in JMP through multistep processes. For example, AIPW can be performed so that the treatment and response models are fit using machine learning tools like the XGBoost and Torch add-ins, or by using the generalized regression and GLMM platforms.
This blog was co-authored with Safiya Sirota (@safiya_jmp), who contributed to this work during her internship with us.
References
Abadie A, Imbens G W (2006). “Large sample properties of matching estimators for average treatment effects”. Econometrica, 74(1), 235–267. https://doi.org/10.1111/j.1468-0262.2006.00655.x
Benedetto U, Head, S J, Angelini, GD, Blackstone, EH (2018). “Statistical primer: Propensity score matching and its alternatives”. European Journal of Cardio-Thoracic Surgery, 53(6), 1112–1117. https://doi.org/10.1093/ejcts/ezy167
Elze MC, Gregson J, Baber U, Williamson E, Sartori S, Mehran R, Nichols M, Stone GW, Pocock SJ (2017). “Comparison of propensity score methods and covariate adjustment: Evaluation in 4 cardiovascular studies”. Journal of the American College of Cardiology, 69(3), 345–357. https://doi.org/10.1016/j.jacc.2016.10.060
Glynn AN, Quinn KM (2010). “An introduction to the augmented inverse propensity weighted estimator”. Political Analysis, 18(1), 36–56. https://doi.org/10.1093/pan/mpp036
Scharfstein DO, Rotnitzky A, Robins JM (1999). “Adjusting for nonignorable drop-out using semiparametric nonresponse models”. Journal of the American Statistical Association, 94(448), 1096–1120.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.