Logical inference about neural networks as models of brain activity and behavior
– a saga in 8 or 9 tweets –
by me, soon to be vastly improved by @o_guest /0

We often reason about what models represent, learn, or ‘know,’ based on performance on a task, or whether the model approximates human behavior/brain activity. Then sometimes we also try to infer what algorithm the human mind and brain instantiates based on this. /1

The arguments roughly go like this: “if the model approximates human behavior/brain data, then it does what people do” (read: it achieved this by doing what people do or having what people have). This appears to be modus ponens. But it isn’t! I’ll get back to that later. /2

Modus ponens: “if the model does what people do, then it will approximate human behavior”. That is good. But if we grant that, why not modus tollens? “if the model does what people do, then it will approximate human behavior; it does not, ergo it didn’t do what people do” /3

We rarely see modus tollens in the literature. Rather, the argument tends to go “the model did not get all the data humans get, that's why it behaved differently.” (solidarity to @GuillermoPuebl6 who dealt w/ this a lot; & see important work by @IrisVanRooij @MarkBlokpoel ) /4

But the same “not-the-same-data-as-humans-get” worry holds for all modus ponens cases. Really it holds for all modelling. So why is modus ponens accepted as valid, but not modus tollens? /5

Actually what we are doing is affirming the consequent: “if the model approximates human behavior, then it does what humans do” /6

But there are quite literally infinite ways it could produce the same performance… that have nothing to do with what humans do. Thus, I give you the fallacy of AFFIRMING THE CONSEQUENT, and not modus ponens /7

Modus ponens
If models do what humans do, then they will approximate human behavior.

Modus tollens
If models do what humans do, then they will approximate human behavior; but if they do not, then they do not do what humans do. /8

Affirming the consequent
If models approximate human behavior, then they do or have what people do or have.

AKA “modus don’tens,” or the epic “modus trollens” @o_guest /9