Greg Yang, Discover and Read the Best of Twitter Threads by @TheGregYang

Greg Yang

TheGregYang

1/ The famous Collatz conjecture asks if repeating the following on any number always ends in 1: if the number is even, half it; if odd, triple it & add

1/4 WTF guys I think I broke ML: loss & acc 🡅 together! reproduced here https://github.com/thegregyang/LossUpAccUp. Somehow good accuracy is achieved *in spite of* classic generalizn theory (wrt the loss)

1/ I've taken positional embedding in transformers for granted, but now looking at it: is the main reason for the unnatural sin/cos formulation the "nice" autocovariance structure? NLP twitter help