Read on Twitter
We talked about Hattie’s use of the term “effect sizes” in class last night and I woke up still annoyed by this ridiculous “zone of desired effects” image 1/10
Bob Slavin has a post critiquing Hattie’s approach here https://robertslavinsblog.wordpress.com/2018/06/21/john-hattie-is-wrong/">https://robertslavinsblog.wordpress.com/2018/06/2... 2/10
And there’s a counter point here https://blogs.edweek.org/edweek/finding_common_ground/2018/06/hattie_isnt_wrong_you_are_misusing_his_research.html">https://blogs.edweek.org/edweek/fi... that argues that “we don& #39;t like large effect sizes that are over a 1.00.” 3/10
…but it’s not about “liking” large effect sizes or not. It’s about whether we believe these are plausibly causal effects 4/10
Guideline #1 from @MatthewAKraft: 1. Results From Correlational Studies Presented as Effect Sizes Are Not Causal Effects https://journals.sagepub.com/doi/full/10.3102/0013189X20912798">https://journals.sagepub.com/doi/full/...
The “zone of desired effects” is built on a lie. Correlation is not causation. Causal effects in education are simply not this large. 6/10
Correlation evidence is not bad in and of itself! It can point us in the direction of promising interventions. But in a time when we& #39;re facing budget cuts, it would be really beneficial to know which programs *cause* improved student outcomes 7/10
There are lots of more rigorous meta-analyses out there. I checked out a few, and came up with this. There are things that work! But the effect size is NEVER as large as Hattie’s “zone of desired effects” 8/10
BTW this is pretty consistent with what @BethSchueler https://www.educationnext.org/summer-vacation-academies-narrow-coronavirus-learning-gaps-springfield/">https://www.educationnext.org/summer-va... and @MatthewAKraft https://www.brookings.edu/blog/brown-center-chalkboard/2020/05/21/getting-tutoring-right-to-reduce-covid-19-learning-loss/">https://www.brookings.edu/blog/brow... have been calling for 9/10
This next school year is going to be extraordinarily difficult, and I hope that we bring strong evidence to address the challenges we face. 10/10
Citations for the figure above (for @Larryferlazzo, and anyone else interested) -
Reading interventions https://www.tandfonline.com/doi/full/10.1080/19345747.2019.1689591
Tutoring">https://www.tandfonline.com/doi/full/... #fromrss">https://www.nber.org/papers/w27476 #fromrss
Head">https://www.nber.org/papers/w2... Start https://journals.sagepub.com/doi/10.3102/0162373712462453
Grades">https://journals.sagepub.com/doi/10.31... https://www.tandfonline.com/doi/pdf/10.1080/01443410.2019.1659939?needAccess=true
Homework">https://www.tandfonline.com/doi/pdf/1... https://pdfs.semanticscholar.org/527e/698c21e1d853900661043f7f7bb71622971e.pdf
11/10">https://pdfs.semanticscholar.org/527e/698c...
Reading interventions https://www.tandfonline.com/doi/full/10.1080/19345747.2019.1689591
Tutoring">https://www.tandfonline.com/doi/full/... #fromrss">https://www.nber.org/papers/w27476 #fromrss
Head">https://www.nber.org/papers/w2... Start https://journals.sagepub.com/doi/10.3102/0162373712462453
Grades">https://journals.sagepub.com/doi/10.31... https://www.tandfonline.com/doi/pdf/10.1080/01443410.2019.1659939?needAccess=true
Homework">https://www.tandfonline.com/doi/pdf/1... https://pdfs.semanticscholar.org/527e/698c21e1d853900661043f7f7bb71622971e.pdf
11/10">https://pdfs.semanticscholar.org/527e/698c...
Formative assessment https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1745-3992.2011.00220.x
PD">https://onlinelibrary.wiley.com/doi/abs/1... (reading) https://www.tandfonline.com/doi/pdf/10.1080/19345747.2019.1670884?needAccess=true
Instructional">https://www.tandfonline.com/doi/pdf/1... coaching https://journals.sagepub.com/doi/10.3102/0034654318759268
PD">https://journals.sagepub.com/doi/10.31... (math/sci) https://www.tandfonline.com/doi/pdf/10.1080/19345740802641527?needAccess=true
Vocab">https://www.tandfonline.com/doi/pdf/1... instruction https://www.tandfonline.com/doi/full/10.1080/19345740802539200
12/10">https://www.tandfonline.com/doi/full/...
PD">https://onlinelibrary.wiley.com/doi/abs/1... (reading) https://www.tandfonline.com/doi/pdf/10.1080/19345747.2019.1670884?needAccess=true
Instructional">https://www.tandfonline.com/doi/pdf/1... coaching https://journals.sagepub.com/doi/10.3102/0034654318759268
PD">https://journals.sagepub.com/doi/10.31... (math/sci) https://www.tandfonline.com/doi/pdf/10.1080/19345740802641527?needAccess=true
Vocab">https://www.tandfonline.com/doi/pdf/1... instruction https://www.tandfonline.com/doi/full/10.1080/19345740802539200
12/10">https://www.tandfonline.com/doi/full/...
