Forget the Hollywood version of AI where genius engineers unlock new worlds through groundbreaking code. The real history of artificial intelligence tells a different story—one where every major leap came not from new algorithms, but from new data. From computer vision breakthroughs to the rise of large language models, data has always been the silent protagonist. MIT and Stanford may develop brilliant architectures, but without the right dataset, nothing moves.
The Four Major Turning Points in AI Were All About Data
Look closely at the defining moments in AI's modern history, and a clear pattern emerges: they were all triggered by the discovery or release of a new dataset—not a new idea.
2012 – AlexNet and ImageNet: The rise of deep learning in vision was powered by ImageNet, a dataset of over 14 million labeled images. Without it, the GPU-powered model would have gone nowhere.
2017 – Transformer and the Internet Text Corpus: The transformer model architecture only became revolutionary because it could be trained on massive amounts of internet text. The data made the model.
2022 – RLHF and Human Feedback: Reinforcement learning with human feedback gave AI a new edge—human preferences. By training models on data that reflected how humans judge quality text, large language models became significantly more useful.
2024 – Reasoning Models and Verified Outputs: The latest breakthroughs in reasoning models rely on structured, verifiable data from tools like calculators, compilers, and symbolic checkers.
the proposed “Moore’s Law for AI”. (by the way, anyone who thinks they can run an autonomous agent for an hour with no intervention as of April 2025 is fooling themselves)
We Are Building 2025 AI with 1990s Technology
The techniques underlying today’s AI systems are not new.
Supervised learning still relies on cross-entropy loss functions developed in the 1940s.
Reinforcement learning is based on policy gradient methods and frameworks introduced in the 1990s.
Even transformer-based models are just new arrangements of older mathematical components.
The truth is, we’re not inventing new methods—we’re reusing old methods on new data.
A Harsh Truth: Data Quality Sets the Ceiling
No matter how elegant the architecture, a model cannot outperform the data it learns from. This has been proven time and again.
A recent research effort spent a year developing a new architecture to replace transformers: the Simple State Space Model (SSM).
When trained on the same dataset, the performance was nearly identical.
The takeaway? Architectural novelty cannot compensate for stagnant or limited data.
Better data beats better math. Every time.
The Next AI Revolution Will Come from YouTube, Not Papers
If you're looking for the next big leap in AI, don’t look to a whiteboard—look to YouTube.
Every minute, over 500 hours of video content is uploaded to the platform.
This visual content contains layers of information that text lacks—intonation, physical dynamics, social context, emotion.
If Google fully unlocks YouTube as a training source, it could usher in the next wave of multimodal AI that thinks, sees, and feels more like a human.
Robotic sensor data from physical interactions in the real world is another powerful data source waiting to be mined.
Most Researchers Are Wasting Their Time
The majority of AI researchers are focused on tweaking models. This is a mistake.
While they chase marginal gains through architectural novelty, the real breakthroughs come from discovering new sources of rich, high-signal data.
OpenAI didn’t invent new math. They trained on the entire internet and refined it with human feedback.
DeepMind’s most promising recent projects are not Go or StarCraft—they're large-scale simulations with diverse sensorimotor feedback.
The bitter lesson is this: compute and data always win. Fancy models do not.
The Smartest AI Startups Are Not Building Better Algorithms—They’re Hoarding Better Data
If you’re an AI startup, forget the temptation to build the next big model. Focus on capturing, curating, and protecting unique datasets that others can’t access.
Venture capital should prioritize teams that own or can generate proprietary data—whether from industrial workflows, healthcare conversations, or real-world interactions.
The next big thing in AI will not come from better code, but from deeper, richer, more exclusive data pipelines.
The central question for the next decade of AI is not “What can we build?”
It’s: “What data do we have that no one else does?”
His column is inspired from Jack Morris’s blog
https://blog.jxmo.io/p/there-are-no-new-ideas-in-ai-only
