Read on Twitter
Since various SW benchmarks are going around today... A short thread on why I use #rstats.
Put simply, it offers by far the fastest & most efficient tools for the work I do (i.e. mostly data wrangling & applied econometrics).
Put simply, it offers by far the fastest & most efficient tools for the work I do (i.e. mostly data wrangling & applied econometrics).
(Disclaimer: This thread is *not* tying to get you to change from your preferred SW. You should use whatever you feel comfortable with. But I will try to highlight some objective facts that matter to me.)
For data wrangling, nothing comes close to consistently matching the performance of #rdatatable. Benchmarks here: https://h2oai.github.io/db-benchmark/
(The tidyverse obviously provides another extremely rich data wrangling framework in R & comes w/ its own set of awesome features: SQL, Spark, Arrow etc. integration.)
If you wondering about Stata (not incl. in the above benchmarks), see
https://github.com/matthieugomez/benchmark-stata-r, or
https://grantmcdermott.com/2020/06/30/reshape-benchmarks/
Bottom line: even if I grant you gtools (which you should install), an MP license ($$), and constrain the no. of cores that R uses, R is consistently faster.
https://github.com/matthieugomez/benchmark-stata-r, or
https://grantmcdermott.com/2020/06/30/reshape-benchmarks/
Bottom line: even if I grant you gtools (which you should install), an MP license ($$), and constrain the no. of cores that R uses, R is consistently faster.
For fixed-effect regressions, {fixest} is insanely quick... as much as a 100x faster than lfe and reghdfe (both great packages in their own right). https://github.com/lrberge/fixest/
And... there’s more! It also supports non-linear models (logit, etc.)
And... there’s more! It also supports non-linear models (logit, etc.)
Or, maybe you’re interested in LASSO. To the best of my knowledge, the {biglasso} package is easily the fastest and most memory efficient implementation. https://github.com/YaohuiZeng/biglasso
A quasi-related issue is code concision/syntax. This is veering off the “objective” path (I don’t have detailed stats) but I can only smile at claims that R requires more lines of code than, say, Stata. The opposite is almost always true IME. https://twitter.com/causalinf/status/1154433269355753473
Fwiw, compare the following bits of code. This is literally the most recent bit of Stata code that I rewrote in R.
Again, though: concision isn’t necessarily a goal unto itself. Good code is code that you (and your collaborators) find easy to write and understand. There’s nothing wrong with writing more verbose code that achieves these goals. Code shaming is despicable IMO.
In summary, I use #rstats because it offers the best tools for *my* needs. The awesome community and zero price tag don’t hurt either ;-)
Your needs and tolerance to learn a new SW language may differ. But you should know that performance loss is *not* a reason to avoid it. /fin
Your needs and tolerance to learn a new SW language may differ. But you should know that performance loss is *not* a reason to avoid it. /fin
