1/ So I spent a day trying to understand GPT-3: what it is and how it works, from the pov of a layperson.

Compiling my learnings here. https://abs.twimg.com/emoji/v2/... draggable="false" alt="👇" title="Rückhand Zeigefinger nach unten" aria-label="Emoji: Rückhand Zeigefinger nach unten">

Hope it helps a few to get a basic understanding of GPT-3 and how it works.

It& #39;s time for a

GPT-3 MEGATHREAD! https://abs.twimg.com/emoji/v2/... draggable="false" alt="🔥" title="Feuer" aria-label="Emoji: Feuer">https://abs.twimg.com/emoji/v2/... draggable="false" alt="🔥" title="Feuer" aria-label="Emoji: Feuer">https://abs.twimg.com/emoji/v2/... draggable="false" alt="🔥" title="Feuer" aria-label="Emoji: Feuer">

2/ Disclaimers:

I& #39;ve dabbled with basic linear/logistic regression and Bayesian models earlier in my career. My knowledge is still pretty amateurish, and this thread isn& #39;t meant to be *precise* as much as its meant to layperson-friendly peek into the blackbox that& #39;s GPT-3.

3/ Having said that, if you find any grievious mistakes, please reply to the thread and I& #39;ll add the corrections to the thread.

All quotes used in this thread are from the GPT-3 whitepaper.

4/ To start off with, let& #39;s see how any deep learning ML model "learns."

5/ There are essentially two components to a machine learning model: PARAMETERS and DATA.

The model identifies a set of PARAMETERS that help it to identify and predict the nature of any given data. It learns these parameters this using the DATA it is trained on.

6/ A analogy, we identify a cat by using parameters like

does it have the shape of a cat?
features of a cat?
color of a cat?
do others call it a cat?
etc.

And we can do this based on our training dataset — what we already know about cats from our past.

7/ For most of us, that would be something we learned as kids:

that this is a cat and this isn& #39;t a cat.

We often don& #39;t feel this parametric-reasoning happening everytime we see a cat because it happens subconsciously, in realtime.