Read on Twitter
We talked about Hattie’s use of the term “effect sizes” in class last night and I woke up still annoyed by this ridiculous “zone of desired effects” image 1/10
Bob Slavin has a post critiquing Hattie’s approach here https://robertslavinsblog.wordpress.com/2018/06/21/john-hattie-is-wrong/ 2/10
And there’s a counter point here https://blogs.edweek.org/edweek/finding_common_ground/2018/06/hattie_isnt_wrong_you_are_misusing_his_research.html that argues that “we don't like large effect sizes that are over a 1.00.” 3/10
…but it’s not about “liking” large effect sizes or not. It’s about whether we believe these are plausibly causal effects 4/10
Guideline #1 from @MatthewAKraft: 1. Results From Correlational Studies Presented as Effect Sizes Are Not Causal Effects https://journals.sagepub.com/doi/full/10.3102/0013189X20912798
The “zone of desired effects” is built on a lie. Correlation is not causation. Causal effects in education are simply not this large. 6/10
Correlation evidence is not bad in and of itself! It can point us in the direction of promising interventions. But in a time when we're facing budget cuts, it would be really beneficial to know which programs *cause* improved student outcomes 7/10
There are lots of more rigorous meta-analyses out there. I checked out a few, and came up with this. There are things that work! But the effect size is NEVER as large as Hattie’s “zone of desired effects” 8/10
BTW this is pretty consistent with what @BethSchueler https://www.educationnext.org/summer-vacation-academies-narrow-coronavirus-learning-gaps-springfield/ and @MatthewAKraft https://www.brookings.edu/blog/brown-center-chalkboard/2020/05/21/getting-tutoring-right-to-reduce-covid-19-learning-loss/ have been calling for 9/10
This next school year is going to be extraordinarily difficult, and I hope that we bring strong evidence to address the challenges we face. 10/10
Citations for the figure above (for @Larryferlazzo, and anyone else interested) -
Reading interventions https://www.tandfonline.com/doi/full/10.1080/19345747.2019.1689591
Tutoring https://www.nber.org/papers/w27476#fromrss
Head Start https://journals.sagepub.com/doi/10.3102/0162373712462453
Grades https://www.tandfonline.com/doi/pdf/10.1080/01443410.2019.1659939?needAccess=true
Homework https://pdfs.semanticscholar.org/527e/698c21e1d853900661043f7f7bb71622971e.pdf
11/10
Reading interventions https://www.tandfonline.com/doi/full/10.1080/19345747.2019.1689591
Tutoring https://www.nber.org/papers/w27476#fromrss
Head Start https://journals.sagepub.com/doi/10.3102/0162373712462453
Grades https://www.tandfonline.com/doi/pdf/10.1080/01443410.2019.1659939?needAccess=true
Homework https://pdfs.semanticscholar.org/527e/698c21e1d853900661043f7f7bb71622971e.pdf
11/10
Formative assessment https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1745-3992.2011.00220.x
PD (reading) https://www.tandfonline.com/doi/pdf/10.1080/19345747.2019.1670884?needAccess=true
Instructional coaching https://journals.sagepub.com/doi/10.3102/0034654318759268
PD (math/sci) https://www.tandfonline.com/doi/pdf/10.1080/19345740802641527?needAccess=true
Vocab instruction https://www.tandfonline.com/doi/full/10.1080/19345740802539200
12/10
PD (reading) https://www.tandfonline.com/doi/pdf/10.1080/19345747.2019.1670884?needAccess=true
Instructional coaching https://journals.sagepub.com/doi/10.3102/0034654318759268
PD (math/sci) https://www.tandfonline.com/doi/pdf/10.1080/19345740802641527?needAccess=true
Vocab instruction https://www.tandfonline.com/doi/full/10.1080/19345740802539200
12/10
