Yann LeCun’s new venture is a contrarian bet against large language models

You’ve been working on AI long before LLMs became a mainstream approach. But since the advent of ChatGPT, LLM degrees have become almost synonymous with AI.

Yes, we will change that. Perhaps the public face of AI is mostly master’s degree holders and chatbots of various kinds. But the latest is not just an LLM. It’s an LLM plus a lot of things, like perception systems and code that solves certain problems. So, we will see to some extent that LLMs are kind of coordinators in systems.

Beyond MBAs, there is a lot of AI behind the scenes that runs a large portion of our society. There’s the in-car driving assistance software, the fast-spinning MRI images, the algorithms that drive social media — it’s all artificial intelligence.

I’ve been frank in saying that LLMs can only take us so far. Do you think LLMs are overrated these days? Can you summarize for our readers why you think LLM degrees are not enough?

There’s a feeling that it’s not overrated, which is that it’s very useful for a lot of people, especially if you’re writing script, doing research, or writing code. LLMs handle language really well. But people have had this delusion, or delusion, that it’s just a matter of time until we can bring them up to the level of human intelligence, and that’s simply wrong.

The really hard part is understanding the real world. This is the Moravec paradox (a phenomenon observed by computer scientist Hans Moravec in 1988): what is easy for us, such as perception and navigation, is difficult for computers, and vice versa. LLMs are limited to the world of discrete text. They cannot really think or plan, because they lack a model of the world. They cannot predict the consequences of their actions. That’s why we don’t have a home robot that’s as flexible as a house cat, or a truly autonomous car.

We will have artificial intelligence systems with human-like intelligence, but they will not be built on master’s degree holders, and that will not happen next year or two years from now. It will take some time. There are major conceptual breakthroughs that need to happen before we have AI systems with human-level intelligence. This is what I’ve been working on. This company, AMI Labs, is focused on the next generation.

And your solution is global models and JEPA architecture (JEPA, or “Joint Embedding Predictive Architecture,” is a learning framework that trains AI models to understand the world, created by LeCun while at Meta). What is an elevator pitch?