I’ve mentioned this study before. It’s by Levitt and it’s also the kind of study I always think of with him - kind of bizarre, somehow still about economic theory, well identified, extremely creative. https://pdfs.semanticscholar.org/5cd3/ce4680f6c40fce314bcba200a7ad59ac327d.pdf

I emailed Levitt and asked if I could have the data so we could replicate it in my Mixtape so as to illustrate IV. Basically, I think through the FREAKONOMICS podcast, Levitt solicited participation in an experiment about indecision and I think 20k volunteered

People who self selected into the experiment were drawn to the experiment precisely bc they were stuck. Should they quit their job? Should they break up or even divorce? To milder things like Should they get that different hair style? They were selected based on indecision is key

Not everyone gets stuck, so the study has no external validity for the broader population partly for that reason. But before I talk about the papers limitations, here’s the experiment: Levitt flipped 20,000 coins for each respondent. Heads, go through with decision. Tails don’t.

I like this study for many many reasons. I love it pedagogically bc I think this is something a lot of people can identify with. How many people stay too long in a toxic relationship? Answer: A LOT OF PEOPLE. Even when they know they should exit, they struggle to do so.

So pedagogically it is a very useful example if only bc the typical student will immediately identify with it. Secondly, it’s going to aggravate many students which I always find useful tbh bc the students start talking with me, the subject matter and importantly, each other.

But let’s talk about the more scientific elements. Levitt records whether they went through with it, demographics, the Coin outcome, baseline outcome measures, and then in a subsequent survey 6 months later, their overall subjective happiness.

So what happened? First it appears there is a strong first stage. I’m just trying to read the table for now, but I think there’s a 11 percent increase in making a serious decision if heads and a larger compliance with heads for less serious/challenging choices

He then does a 2 and 6 month follow up. I’ll focus on the 6 month. He reports both OLS and 2SLS results, and as this is a randomized decision experiment I’ll focus on the last column of the 6mo table. Standard errors in parenthesis. 6mo later compliers report more happiness

They report it for the very serious things like quitting their job, or even breaking up with someone! He goes through and does the necessary sensitivity analysis including attrition analysis. I’ll let you read those sections and see what you think. I’d like to comment now.

First, this is a very salient illustration of the LATE parameter. He is only identifying effects for the compliers which here is only those who willingly agreed to do what heads said to do. That’s an important qualification, but honestly, I find it amazing we get even that.

Second, this instrument fits one of the important often neglected assumptions of IV with heterogeneous treatment effects - independence. Which is why the best instruments are randomized. This is a coin flip and it is independent of potential outcomes by design.

Third, there is a strong first stage which can be empirically verified. I didn’t see a first stage F test but I was hoping to get the data to look at that more closely. Fourth is the exclusion restriction though

The exclusion restriction for this design means that heads cannot affect 6mo out happiness except via its effect on the persons choice. Can it be argued that seeing a virtual coin come up heads makes you happier regardless of the decision made?

I don’t know, but it’ll definitely prompt good discussion in class to say the least and it’ll center that discussion around the exclusion restriction which sometimes is too subtle for first time students hearing about IV. (I can’t tell a plausible argument why heads matter...)

But then there is the actual outcome. Happiness - such a crucial outcome for many people. Now happiness is not the same as utility (something lost on many), and happiness isn’t imo the goal of the meaningful life. We are often called to choices that do not increase happiness

Including the things on this list. But, I think at the same time it is quite easy to tell yourself stories that lead you to remain in truly toxic, terrible situations and that can be done by moralizing the suffering into a virtue - which is truly a dangerous thing to do

But what does it mean? I’ve talked with several economists and they all interpret it differently. Levitt seems to favor a behavioral explanation involving status quo biases like loss aversion. Maybe his evidence for it is good but subtle and I need to study it more closely.

I can’t remember what @economeager said when I told her about it, but we were talking about it in terms of toxic relationships and we both seemed to think it had some relevance even if we didn’t fully understand what it meant

Another friend thought it wasn’t loss aversion so much as Levitt was taking the choice away and that was a relief for many. Iow, it wasn’t irrational weighting or downside risk as much as how much they hated the choice itself and Levitt relieves them of that burden.

I find many explanations plausible, which again is why this is both pedagogically a useful IV example for students as well as imho an interesting scientific study that honestly matters for difficult lives. When do we do something hard? And should happiness even matter?

Anyway, I love this paper and I’m trying to get the data from Levitt. He’s a saint when it comes to sharing his data, but he’s got an R&R and wants to wait til it goes through. I have a February deadline so it may not work out, but I think it’ll feature in the mixtape

Oh one more thing - monotonicity. This is an example where unlike judge fixed effects it’s a little challenging to tell a coherent story of a monotonicity violation. A violation would require some people with heads not complying but complying if tails. What weirdos!

Last thing - even IV skeptics like @ez_angus have to grant this design!