Read on Twitter
Well I guess Mr Dr Ranty Methods Curmudgeon is back because I& #39;m about to go off on people muddling critiques of propensity score matching 1/ https://twitter.com/PHuenermund/status/1251953408497668096">https://twitter.com/PHuenermu...
PSM is a method with a lot of critiques. They fall into 3 basic camps: 1) propensities are bad 2) matching is bad 3) both propensities and matching are inadequate no matter how you use them b/c omitted variables 2/
The propensities are bad argument is best articulated by King & Nielsen (2019) where they argue that you need matching on variables instead of propensities. Collapse to a propensity and you get wonky geometry & inappropriate matches. 3/
They advocate coarsened exact matching (CEM) instead of propensities. It has all the bad things about matching but solves a specific and real problem with propensities collapsing all the data into one dimension when maybe you shouldn& #39;t do that. 4/
Their critique is really only about the fact that some things are crucial to match on exactly and PSM messes that up. They specifically say that you could do a hybrid between CEM (on the crucial things) and propensities but that seems to get lost in the marketing. 5/
The critiques of matching generally can be found in Hernan & Robins& #39; Causal Inference book where they strongly advocate for propensities used in IPW instead of matching. It saves cases and is much more flexible. Their argument in general is a strong one. 6/
For matching of any kind you basically just need a ton of cases and you end up throwing away lots of data. The matching algorithms can be weird and complicated. You can avoid the whole issue just by using weighting (IPW) 7/
In either PSM or IPW you& #39;re building the same propensity score. You& #39;re just using the first to select a subset of cases whereas the second builds you weights for all the cases. Ideally, they& #39;ll produce the same answers. 8/
The third critique is that none of it works b/c you can& #39;t condition on unobservables. If you don& #39;t have it in your model then it doesn& #39;t matter if you use propensities or matching or whatever b/c it& #39;s still biased. This is the common critique in econ 9/
There are partial answers to this critique but there& #39;s no "solution" to not having something crucial when you need it. There& #39;s also no telling if you do or don& #39;t absent randomization (and in practice even then it& #39;s not certain) 10/
You can use propensities with latent variables to match on some unobservables with something like multilevel IWP or PSM. This is becoming more common but is still pretty rare. CEM won& #39;t work in this context 11/
You specify a multilevel logit instead of a regular logit and build propensities based on a fully saturated model with random intercepts and slopes. You can easily change to machine learning options with classification algorithms & build propensities that way too 12/
You can use sensitivity analysis with propensities the same way you would with linear regression. You& #39;re basically measuring how big the effect of the omitted variables would need to be to offset what you& #39;re seeing. This is a growing area as well 13/
Lastly you can use double robustness techniques like combining PSM/IPW with something like a DiD. You use two imperfect methods together to try to offset one another& #39;s assumption violations. It& #39;s not perfect but it can often help. Stuart et al 2015 is great for this 14/
The point of all of this is just to say that it& #39;s important to be aware of what exactly your problem is with PSM. People have a vague sense that it& #39;s "bad" w/o a real understanding of why and they misattribute problems. Some of these things have fixes and some don& #39;t 15/
Both propensities and matching can be valuable tools in general and throwing them away b/c of a vague sense that they are unfashionable is terrible for science. No method is perfect or universally applicable & we are stuck trying to make these things work as well as we can. 16/
