Live reading @IrisVanRooij and @giosuebaggio’s recent commentary “Theory development requires an epistemological sea change” on @EikoFried’s article on lack of theory building.

They start by acknowledging the pluralism in theoretical reform proposals. That’s a great point to bring up bec there is no one fool-proof approach to theory and different approach may suggest different paths. We’ll find out what the authors mean by “epistemological sea change”.

Here’s one example of that pluralism. This section made me gasp a bit bec in our work, we use statistical modeling to capture some aspect of the data generating mechanism (vs. to model relations between observed variables). i.e., we use stats theory to develop scientific theory.

So this dichotomy does not match my own paradigm for theory development. But I’ll need to come back to that later. For now, I move on to the author’s points of agreement with the target article’s approach.

1. What theory is not: (Psychological) theory is not just a set of statistical models.

I agree w this in the sense that at the min we need mapping btw scientific phenomena of interest and statistical models. Stats alone cannot inform our inference on natural/social phenomena.

However, the authors say “Statistical models […] do not, by design, specify the process that generates the behavior of the system” which I’m not sure I get. I can think of two potential sources for my confusion: Either a very specific set of applications is meant by “statistical

models” or something other than observables is meant by the “behavior of the system”. Maybe there are other explanations as well but those are the only two that occur to me. Anyway I digress. Next we should find out what more we need besides statistical models.

2. Theory needs formalisation

Hmmm I scratch my head once again. Why does statistical theory not provide the desired formalism? This part is less clear to me as no particular explanation jumps to my mind readily.

3. Most theory in psychology is weak

Well alright. No contentions here!

4. Poor inference from lack of theory

“Current psychological science, and its associated epistemology, have largely focused on just one type of uncertainty that potentially marrs the robustness of inferences to and from data: statistical uncertainty,

e.g., as linked to probabilities of accepting or rejecting hypotheses about effects in a given population, based on data from a random sample.”

Ah ok a theme emerges. I do have a different view of where things go wrong: widespread model misspecification, sampling problems, and

a misuse and abuse of statistical procedures in the absence of proper probabilistic modeling. So poor inference, yes, but the “from lack of theory” part should also encompass the lack of statistical theory as opposed to being evaluated as a separate problem.

That’s enough food for thought for now. I’ll continue my live reading tomorrow. G’nite for now!

Oops I forgot to link the article: https://psyarxiv.com/rnw9q/