If you’re looking for something to watch this late Saturday evening, join me in watching this documentary on Claude Shannon.

https://vimeo.com/315813606 
Password: Shannon-ISIT (valid this weekend)
Going to live tweet somethings because of the Shannon fanboy I am 😄
Okay, first impressions: The documentary itself (title: “THE BIT PLAYER”) is super well produced. In genre, it reminds me of the SECRETS OF THE SURFACE, the movie about the amazing Maryam Mirzakhani, which everyone should watch too.
http://www.zalafilms.com/secrets/#about 
Okay, back to Shannon. Turns out he’s quite a bit of a tinkerer and inventor of gadgets and such. You will be forgiven to think he was a theoretician, because of his gigantic contributions there.
The man. “What if it were like this”
Communication was happening before Shannon and things were super unreliable. Then Shannon came along and changed everything. Reminds me of today’s deep learning. Who will be the Shannon of deep learning?
Not surprisingly, noisy channel model comes very early in the movie.
Watching this you get the impression Shannon was more of a Tesla building stuff in his backyard than a mathematician at his desk scribbling proofs. As a child, he built a telegraph machine and rightly reminds everyone in the movie that Morse invented the “bit”.
Hah! Who knew Edgar Allen Poe did information theoretically optimal decoding in his short story? Now I want to read it. Morse, Turing, Shannon, they were all interested in finding patterns. See a pattern?
“Play infused everything he did”
“Nothing was sacred”

#TombstoneGoals
His award winning masters thesis: connecting Boolean algebra to electric switches. No wonder it’s called the “most important masters thesis” of all time.
Wait what? Back then, Boolean algebra was called “queer algebra”! 🤓🤗
Interesting titbit from the movie: you had to learn French to graduate from your PhD program back then. 🤔🥖
For his PhD, he switched to biology (!) to apply mathematics to genetics.
“You know DNA might be the ultimate information storage device” ... can’t wait to move away from solid state.
“You could represent different genetic possibilities mathematically in matrices. Then you can apply matrix algebra to make predictions”

Can we all agree that deep learning is just rebranding for a new age and new set of tools?
A little human moment. He’s talking about Einstein, Von Neumann, and the IAS and drifts off into either a reverie or something. Interviewer after a bit, “Dr. Shannon?” (wakes up startled)
Shanon finds that people at IAS are more interested in pure mathematics and not so interested in practical things, stops working, gets depressed, and ends up in a divorce.

Ever feel like you don’t want to go work and you don’t like the work you signed up for? You’re not alone.
1945 is the first time the bigram “information theory” comes in written word. Interviewer asks, “did your information theory work come from your cryptography work?”
Shannon: it all happened at the same time. How you make some thing intelligible and how you make something unintelligible are two sides of the same coin.

(brainwaves)
He works on his universal theory of communication silently for 10 years. Alone. Because he’s not sure if it’s making sense and people around him don’t seem to be asking the right questions.

Think of the psychological impact of doing something like that!
How do you measure information, like energy? His mega conclusion: “meaning of the content is irrelevant”. Everyone before him could not separate the meaningness of things when considering “information”.
Tying “information” to removal of uncertainty and quantifying that created new ways of approaching the topic. This leads to a counterintuitive idea: Gibberish contains great information than great literature.
This leads to compression. The connection is obvious. Optimality of compression comes being able to formulate information as reducing uncertainty.
Shannon Limit is a groundbreaking idea, like the speed of light is a constant. How do we code information to reach the Shannon Limit? Error Correcting Code.
When the Universal Theory of Communication came out, it got instant recognition. This is clearly gratifying even to the viewer. Thank god he didn’t end up like Tesla (even if you knew how this story ends).
Everyone is going crazy about this work. It is thought information theory is the answer to everything. Shannon gets fed up and writes a bulletin to curb that. Can’t think of any other examples in science like this. Do you?
Out of nowhere there is a maze solving electro-mechanical contraption that Shannon built appears in the movie.

“It learns by exploration” Sounds familiar?
Now the focus is on his second wife. This part of the movie is seriously cool. She’s a mathematician, and she worked as a “computer” back when only women did computing. She was also into weaving. Ada! Too bad they didn’t keep it going, but I can live with turning back to Shannon.
Now we’re in the MIT part of his life. He’s again less interested in working. Stays home, rides unicycles, and juggles.

What is it that juggling attracts so many math folks? My favorite juggler mathematician is Ron Graham, BTW. Also, big fan of his Concrete Mathematics book.
He worked on applying the noisy channel model to trade stocks. This is way before Robert Mercer, Renaissance, etc. Shannon is likely the world’s first quant!
Shannon has this idea that chess can be solved using computing. But he doesn’t have enough compute at that time. So what does he do? He builds ENDGAME. He simplifies the problem to solve only the last 7 moves, and is able to build something viable.
This idea of simplify and execute appears throughout Shannon’s life and work. I think it’s the mathematics training.

Digression: This is also, not incidentally, one of the best ways to build startups. Many failures can be postmortemed to lack of that ability.
Back to Shannon. The genius exudes in so many directions. Now we are talking about fancy wearable computer (!) he built to p0wn casinos at blackjack. The details are crazy and you must watch this part of the movie. It brings a sense of wonder, play, and adventure to life.
Finally, in 1993 the Shannon Limit is reached. Well almost. For details read the now famous Turbo Codes paper: https://ieeexplore.ieee.org/document/397441 

Side note: IIRC, the first time the paper was submitted it was rejected because it was “too good to be true”.
Oh, @ericschmidt makes an appearance too 😃👋🏼
I think now we are getting into his life and body. He has already developed Alzheimer’s and a gradual dimming of a once sharp mind. Almost feels like too bad his life was only this long even if he lived a full life. Movie slides into credits.
By the end of the movie I’m invested in his well-being even if I know it’s not him and just an actor, and I know the ending already.
It was fun watching this with you all and recording this live. FIN.
If you are inspired by this and want dig deeper into information theory, try this classic by Cover & Thomas https://www.amazon.com/dp/B00HTK9U28/ref=cm_sw_r_cp_api_i_yEf-EbH0R6BCX
And, finally, if you missed watching it over the weekend, I have some good news for you from the folks at @ieee_itsoc https://twitter.com/ieee_itsoc/status/1277559658652340224?s=21 https://twitter.com/ieee_itsoc/status/1277559658652340224
You can follow @deliprao.
Tip: mention @twtextapp on a Twitter thread with the keyword “unroll” to get a link to it.

Latest Threads Unrolled: