AI can write code, diagnose diseases, and speak fluently in multiple languages.
But it still can’t intuitively understand gravity the way we do. Or remember what you told it last week without constant reminders. Or plan its next five steps in an unpredictable environment.
Why?
Because current large language models (LLMs) aren’t built to reason, remember, or interact with the real world. They are built to predict.
They don’t experience the world; they model patterns in it. They don’t think in concepts; they generate tokens.
LLMs will continue to thrive in domains that involve communication, productivity, content creation, and knowledge work. But there’s a growing class of challenges in robotics, care work, logistics, disaster response, and personal assistance that can’t be solved with language alone.
Are LLMs Hitting a Wall?
Language models have gotten us far. But we’re starting to see the edges.
There’s a growing sense among researchers and builders alike: we may be reaching the maximum range of what this architecture can accomplish on its own.
Here’s what LLM Agents struggle with:
Persistent memory: They forget across sessions, tasks, and time. Many agents lack a stable sense of continuity. They don’t build long-term context or evolve their understanding of you or the world in a reliable way.
Generalization to new environments: They perform well in controlled settings but often fail when the environment changes.
Long-term planning: They lose coherence over long sequences and can’t natively remember past interactions unless engineered to do so.
Physical grounding: They have no sense of space, movement, or consequence in the real world.
“Even with fine-tuning, retrieval-augmented generation, and tool use, we’re still fundamentally optimizing a word prediction engine.”
To go further, we need a new architecture, one that incorporates world modeling, embodiment, and multimodal interaction. One that moves from language-only intelligence to embodied intelligence.
Think about how we learn.
We don’t just read. We do. We experiment, fail, and adjust. We learn through our bodies, context, memory, and time. Intelligence is a full-body, full-environment phenomenon, not a disembodied stream of words.
When AI lacks persistent memory, it struggles to personalize experiences, retain feedback, or build on past interactions. Each session starts from scratch, with the same introductions, explanations, and corrections. Users find themselves constantly re-teaching their AI assistant, and that gets frustrating fast.
Persistent memory is what lets humans learn over time, build relationships, and improve with experience. Without it, AI agents stay stuck in isolated tasks that are helpful but shallow.
Human intelligence did not develop by reading millions of sentences from the internet; it developed by living in the world. From infancy onward, humans (and animals) build rich world models: understanding object permanence, learning that crying can bring a caregiver, and figuring out that pushing an object makes it move. This experiential, self-supervised learning forms the basis of common sense.
Scaling LLMs alone won't get us to AGI, or what Dr. Yann LeCun calls Advanced Machine Intelligence (AMI). Simply adding more data, tokens, and GPUs cannot solve the fundamental challenges of physical reasoning, memory, and planning.
Some LLMs are trained on 20~30 Trillion tokens. If each token is represented on average by 3 bytes, for 30 Trillion tokens, that would equate to roughly 100,000,000,000,000 bytes (10¹⁴ bytes).
A child, by age 4, has seen and processed around the same amount of raw visual data (10¹⁴ bytes) just by interacting with the physical world.

So even though the volume of data is similar, the quality, diversity, and depth of the child’s learning experience is vastly richer and better suited for developing advanced intelligence.
This highlights a critical truth: intelligence isn’t about how much data is processed, but how it’s experienced, grounded, and integrated. A child's interaction with the world is multimodal, contextual, embodied, and constantly reinforced through feedback and consequence.
To achieve AGI or AMI, we’ll need systems that do more than predict the next word. They must perceive, remember, reason, plan, and act within environments that are dynamic and unpredictable. This calls for world models that support long-term memory, hierarchical planning, and sensorimotor learning.
What Are World Models?
A world model is a system’s internal understanding of how the world works, how objects move, how cause leads to effect, and how actions change the environment. It’s not about language. It’s about learning from perception and using that knowledge to reason, predict, and plan.
A world model's primary role is to estimate missing information about the state of the world and to predict plausible future states.
It is hypothesized that this accumulated knowledge, learned through observation, forms the basis of common sense. Viewed as a collection of world models, it enables an agent to infer what is likely, plausible, or impossible.
But how do we actually train machines to build and refine such models?
One promising approach is Joint Embedding Predictive Architecture (JEPA), proposed by Dr. Yann LeCun.
Joint Embedding Predictive Architecture (JEPA)
JEPA is a self-supervised learning architecture that trains machines to predict abstract representations of future states, rather than raw data like pixels or tokens.
By modeling how the world changes over time in a compressed latent space, JEPA enables systems to reason about dynamics, anticipate consequences, and build internal simulations all without needing explicit labels or supervised tasks.
This architecture learns to predict future states of the world, frames of video, changes in spatial layout, or the outcomes of actions based on what it perceives and what it does.
The idea is to move from:
Predicting tokens ➝ to predicting structured world states
Consuming language ➝ to integrating perception, action, and memory
Working with static prompts ➝ to engaging in dynamic environments
JEPA models use joint embeddings of multiple modalities (vision, action, proprioception, audio) to create a shared internal space for learning across time.
World Models in the Lab
The most promising early applications of world models will likely emerge in scientific discovery and medical research. Agents can run virtual simulations of molecules and proteins, reason about chemical interactions, or plan multi-step experiments based on prior outcomes. With a well-developed world model, such systems can hypothesize, simulate, test, and iterate across thousands of experimental paths.
In medicine, world-model-based agents could help model the progression of disease, predict treatment outcomes, and even coordinate robotic surgical tasks with adaptive precision.
A 5-year horizon for fully embodied, intelligent agents may be overly ambitious, but 10 years is plausible for scaled, domain-specific deployment. We’re likely to see breakthroughs first in structured environments like labs, factories, and hospitals, where the range of possible inputs and actions is bounded.
What are your thoughts?
If machines could build an internal model of the world like we do, what new kinds of intelligence do you think would emerge? I’d love to hear your perspective!
I hope you enjoyed this article!
If you enjoy my content and want to participate in deeper, ongoing conversations like this, connect with me through my new community here 🎉 :



