#academictwitter It& #39;s time to update how we teach about effect sizes & the policy relevance of research findings.
Here are some resources:
1) I& #39;m hopeful my paper can serve as an accessible introduction to interpreting the importance of effect sizes.
https://journals.sagepub.com/doi/abs/10.3102/0013189X20912798?journalCode=edra">https://journals.sagepub.com/doi/abs/1...
Here are some resources:
1) I& #39;m hopeful my paper can serve as an accessible introduction to interpreting the importance of effect sizes.
https://journals.sagepub.com/doi/abs/10.3102/0013189X20912798?journalCode=edra">https://journals.sagepub.com/doi/abs/1...
2) We need to teach students how to "translate" results into terms that are more widely understood (but still accurate!).
See for example work by Bard & @JohnFPane https://journals.sagepub.com/doi/abs/10.3102/0013189X19848729">https://journals.sagepub.com/doi/abs/1...
See for example work by Bard & @JohnFPane https://journals.sagepub.com/doi/abs/10.3102/0013189X19848729">https://journals.sagepub.com/doi/abs/1...
3) A picture is worth 1,000 words. We need to help students to create better visualizations of research evidence.
Explore work by @EdwardTufte
Or even simple visual guides like this from @Stata
https://www.stata.com/support/faqs/graphics/gph/stata-graphs/">https://www.stata.com/support/f...
Explore work by @EdwardTufte
Or even simple visual guides like this from @Stata
https://www.stata.com/support/faqs/graphics/gph/stata-graphs/">https://www.stata.com/support/f...
4) We need to teach students about opportunity costs and unintended consequences to contextualize policies and programs.
This @J_HumanResource paper by Jepsen & Rivkin on reducing class sizes in CA is one of my favorite examples:
http://jhr.uwpress.org/content/44/1/223.refs">https://jhr.uwpress.org/content/4...
This @J_HumanResource paper by Jepsen & Rivkin on reducing class sizes in CA is one of my favorite examples:
http://jhr.uwpress.org/content/44/1/223.refs">https://jhr.uwpress.org/content/4...
5) We have to teach students to think about program costs and cost-effectiveness:
Check out the great work by @abrooksbowden including:
https://journals.sagepub.com/doi/10.3102/0013189X18814447
A">https://journals.sagepub.com/doi/10.31... classic paper by @douglasharris99:
https://journals.sagepub.com/doi/10.3102/0162373708327524
https://journals.sagepub.com/doi/10.31... href="https://twitter.com/cbcse">@cbcse has a range of great resources.
Check out the great work by @abrooksbowden including:
https://journals.sagepub.com/doi/10.3102/0013189X18814447
A">https://journals.sagepub.com/doi/10.31... classic paper by @douglasharris99:
https://journals.sagepub.com/doi/10.3102/0162373708327524
https://journals.sagepub.com/doi/10.31... href="https://twitter.com/cbcse">@cbcse has a range of great resources.
6) Let& #39;s encourage students to ask - can that program or policy be effective at scale?
This @IESResearch i3 report about attempts to scale promising education interventions is sobering
https://ies.ed.gov/ncee/pubs/20184013/pdf/20184013.pdf">https://ies.ed.gov/ncee/pubs...
This @IESResearch i3 report about attempts to scale promising education interventions is sobering
https://ies.ed.gov/ncee/pubs/20184013/pdf/20184013.pdf">https://ies.ed.gov/ncee/pubs...
7) We also have to reaffirm for students the value of the "gut-check."
Does an effect seem plausible & make common sense?
Is there a theory of action & causal chain of events that would explain the findings?
Check out @NoraEGordon & @clconaway new book for a practical guide:
Does an effect seem plausible & make common sense?
Is there a theory of action & causal chain of events that would explain the findings?
Check out @NoraEGordon & @clconaway new book for a practical guide:
10) Finally, we should build & refine empirical ES benchmarks for specific fields, contexts & outcomes
https://abs.twimg.com/emoji/v2/... draggable="false" alt="🚨" title="Polizeiautos mit drehendem Licht" aria-label="Emoji: Polizeiautos mit drehendem Licht"> @DaveEvansPhD & @Yuan_Fei_ have done just this in a new review of interventions that impact educational outcomes in low- & middle-income countries.
https://drive.google.com/file/d/1HkaK76YJTWGdKWmJYQvVJ3kgW9ieVkO4/view">https://drive.google.com/file/d/1H...
https://drive.google.com/file/d/1HkaK76YJTWGdKWmJYQvVJ3kgW9ieVkO4/view">https://drive.google.com/file/d/1H...
. . . and here is a paper that provides empirical ES benchmarks for metal health and prevention programs.
https://pubmed.ncbi.nlm.nih.gov/30136245/ ">https://pubmed.ncbi.nlm.nih.gov/30136245/...
https://pubmed.ncbi.nlm.nih.gov/30136245/ ">https://pubmed.ncbi.nlm.nih.gov/30136245/...
P.S. here is the open-access working paper version of my Effect Size paper and an excellent summary blog post.
Working Paper
https://scholar.harvard.edu/mkraft/publications/interpreting-effect-sizes-education-interventions
Blog">https://scholar.harvard.edu/mkraft/pu... Post by Matt Di Carlo via @shankerinst https://shankerinstitute.org/blog/interpreting-effect-sizes-education-research">https://shankerinstitute.org/blog/inte...
Working Paper
https://scholar.harvard.edu/mkraft/publications/interpreting-effect-sizes-education-interventions
Blog">https://scholar.harvard.edu/mkraft/pu... Post by Matt Di Carlo via @shankerinst https://shankerinstitute.org/blog/interpreting-effect-sizes-education-research">https://shankerinstitute.org/blog/inte...