In your mental model for how super/general intelligence will occur, there's a very important core axiom.

ML/DL doesn't *create* or *collect* information, it *compresses* information.

It takes a huge number of samples and compresses out the noise, leaving behind latent features

This is *really* important because it means that increasing amounts and types of data is the only path to general and/or super intelligence.

By analogy - ML is an oil refinery - but we need more oil fields and oil drills to get there.

Simulators do the opposite of machine learning - they take latent features (rules of a game) and create noisy samples.

Thanks to P != NP this still creates superintelligent agents for that game - but it won't generalize out-of-domain.

Ex: A chess agent can't read human digits.

The only path towards general super-intelligence is an insane amount of live-updating data about the world.

Wikipedia won't cut it - youtube maybe - but also information on the internet is increasingly behind paywalls and this is likely to increase with digital id tech.

There's also the "embodiment" problem. For those of you familiar - learning an agent based on supervised data of what another agent did doesn't work as well as an agent learning from an environment itself.

Where this will work is for basic physics. Moving around a room - balancing - etc. Because we have hundreds of years of physics research into rules and billions of dollars into incredible simulators thanks to the game industry.

But the real reason is that physics is composable

But there's *so* much more to life than can be learned in a physics engine. And while of course one can argue that life came from a physics engine - the compute required to do taht is insane.