Artificial Intelligence has made remarkable progress in recent years, with large language models like ChatGPT demonstrating capabilities that seemed impossible just a decade ago. However, these achievements, impressive as they are, still fall far short of Artificial General Intelligence – the holy grail of AI development that would match or exceed human-level cognition across many domains.
These remarks by Sam Altman, former CEO of OpenAI, highlight a fundamental limitation in the current approach to developing Artificial General Intelligence (AGI) through the advancement of large language models.
“We need another breakthrough. We can still push on large language models quite a lot, and we will do that. We can take the hill that we’re on and keep climbing it, and the peak of that is still pretty far away. But, within reason, I don’t think that doing that will (get us to) AGI. If super intelligence can’t discover novel physics I don’t think it’s a superintelligence. And teaching it to clone the behavior of humans and human text – I don’t think that’s going to get there. And so there’s this question which has been debated in the field for a long time: what do we have to do in addition to a language model to make a system that can go discover new physics”
Sam Altman – former CEO of OpenAI (Creators of ChatGPT)
There was a lot of speculation recently that he was ousted from OpenAI because they made a breakthrough in AGI which simply isn’t true.
In this post I want to discuss why simply scaling up LLMs isn’t enough to create AGI.
- Prediction vs. Understanding
LLMs, including ChatGPT, are designed to predict and generate text based on patterns learned from vast datasets. While they can mimic human-like responses, their capabilities are fundamentally different from true understanding or reasoning. They don’t possess an internal model of the world or genuine comprehension. LLM’s fundamentally work more like the predictive text that autocompletes your words on Google. - Lack of Novel Discovery
As Altman points out, a key characteristic of AGI is the ability to discover novel concepts or create new knowledge, such as breakthroughs in physics. Current LLMs are limited to reiterating, remixing, or extrapolating from their training data. They lack the capability to innovate or discover something truly new outside of their training scope. - Emulation vs. True Intelligence
LLMs are proficient in cloning human like text responses, but this is not equivalent to possessing intelligence. AGI would entail a broader spectrum of cognitive abilities, including self-awareness, intuition, and the capacity to understand abstract concepts in a way that goes beyond mimicking human text. It needs memory and multithread processes to explore ideas and concepts. - Additional Breakthroughs: Achieving AGI likely requires fundamental breakthroughs beyond just refining language models. It might involve integrating other forms of AI, such as spatial, causal, and logical reasoning, or developing entirely new approaches to machine intelligence that are not currently in the realm of LLMs.
- Safety Considerations
There’s also the aspect of ensuring that AGI, if achieved, aligns with human values and ethics. This is a complex challenge that goes beyond technical advancements and involves creating long term goals and ethics, then ensuring models don’t deviate from them.
While LLMs like ChatGPT represent significant advancements in AI, their nature as text prediction models limits their potential to evolve into AGI.
Achieving true AGI would require multiple breakthroughs that enable models to genuinely understand, reason, and innovate, going beyond the capabilities of current language models.
The LLMs we see today might act like interfaces on top of the processor to bring complex patterns in to human readable input/output but they can’t, in their current state, be considered anything like AGI.
The Current State of AI
Consider a simple physics problem…
While an LLM can recite formulas and even solve equations, it doesn’t truly understand the physical principles at work. It cannot discover new physics theories or generate novel scientific insights, capabilities that would be fundamental to true AGI.
The capabilities require fundamental breakthroughs in how we approach artificial intelligence. Simply scaling up existing language models won’t bridge this gap, we need new paradigms in machine cognition and understanding.
While current AI systems remain firmly under human control, they can be switched off or contained within their operating environments. The development of true AGI would raise unprecedented safety considerations. The global race for AI supremacy further complicates these challenges, as regulatory frameworks struggle to keep pace with technological advancement.
It’s less likely that we see AGI break out of it’s host container and spread like a computer virus than it is we see governments of the world use the technology to fight wars and compete in power struggles.
The path to AGI requires more than technological advancement, it demands a deeper understanding of intelligence itself. Current AI systems, despite their impressive capabilities, represent just the beginning of this journey.
The gap between current AI and AGI remains substantial, but understanding this gap is crucial for realistic expectations and productive development. While today’s AI systems are powerful tools, true AGI requires fundamental breakthroughs in how machines process, understand, and interact with the world. As we continue this journey, maintaining a balance between innovation and safety will be paramount.
The quest for AGI isn’t just about creating smarter machines, it’s about understanding the nature of intelligence itself. As we push these boundaries, we must ensure that our pursuit of artificial intelligence enhances rather than diminishes human potential.
