I recently came across some fascinating insights from Andrej Karpathy's conference presentation about how software is evolving in the AI era and what this means for developers. What struck me most was the observation that students entering the industry right now are experiencing a truly extraordinary time.
The Evolution of Software: From 1.0 to 3.0
The presenter argued that software is undergoing its most fundamental transformation in 70 years, breaking this evolution down into three distinct phases. This classification is remarkably compelling.
Software 1.0represents the traditional code we're all familiar with – developers directly instructing computers through programming languages like C++, Java, and Python.
Software 2.0refers to neural network weights. Instead of writing explicit code, developers curate datasets and train neural networks to learn parameters through data. We've already seen this paradigm shift across many domains.
But here's where it gets interesting: the emergence ofSoftware 3.0. With large language models (LLMs), neural networks have become programmable. Prompts now serve as code for programming LLMs, and remarkably, these prompts are written in plain English.
I find this classification incredibly insightful. Looking at GitHub today, you'll notice repositories aren't just pure code anymore – they're increasingly a mix of English and code. This isn't just a minor change; it represents a fundamental shift in programming paradigms.
Paradigm Shift: Lessons from Tesla Autopilot
The presenter's experience developing Tesla's Autopilot system was particularly illuminating. Initially, the system was predominantly C++ code (Software 1.0) with some neural networks (Software 2.0) handling image recognition.
Over time, as neural networks became more capable and larger, existing C++ code was progressively deleted and replaced by neural network functionality. Even tasks like connecting images from multiple cameras across time became the domain of neural networks.
This observation is crucial. Software 2.0 isn't just a new tool – it's actively "eating" existing software stacks. And now we're witnessing the same phenomenon with LLMs.
LLMs: A New Form of Utility
The presenter's comparison of LLMs to electricity is spot-on. Companies like OpenAI, Google, and Anthropic investing massive capital to train LLMs is analogous to building power grids. Providing this "intelligence" through APIs with per-token pricing mirrors how we pay for electricity.
We've actually witnessed this firsthand when major LLM services went down and people couldn't work. It was literally an "intelligence blackout." Just as unstable power grids cause voltage drops, unstable LLMs make the entire world a little bit "dumber."
LLM Characteristics: Human-like Spirits
The description of LLMs as "people spirits" was particularly striking. Since they're trained on human data, they exhibit human-like psychological characteristics.
LLMs have clear advantages:
- Vast knowledge and memory
- Learning from far more information than any individual human
- Superhuman capabilities in specific domains
But they also have significant cognitive limitations:
- Hallucination phenomena
- Lack of self-awareness
- Inconsistent intelligence (claiming 9.11 is greater than 9.9, or that "strawberry" has two R's)
- Anterograde amnesia (inability to learn new information long-term)
Understanding and working with these characteristics is crucial. LLMs are simultaneously superhuman and flawed, and we need to collaborate with this in mind.
The Era When Everyone Becomes a Programmer
The most revolutionary change is that English has become a programming language. Anyone who can communicate in natural language is now a potential programmer. While software development previously required 5-10 years of learning, that's no longer the case.
The emergence of "Vibe Coding" exemplifies this shift. You can now code based on intuition without knowing exact syntax or structure. Watching children naturally converse with AI while programming shows how dramatically this has lowered the barriers to software development.
The presenter's story about building an iOS app in one day without knowing Swift was impressive. In the past, this would have required days of studying Swift, but now you can start immediately with AI assistance.
Designing Software for Agents
The most intriguing part of the presentation was the idea of designing software for agents. Previously, only humans (through GUIs) and computers (through APIs) consumed and manipulated digital information. Now we have a new category: agents.
Agents are computers with human-like characteristics, and we need to build infrastructure for them. For example:
- Creating `ai.txt` files (like `robots.txt`) to provide domain information to LLMs
- Making documents more readable for LLMs
- Providing interfaces for direct agent interaction, not just APIs
The example of adding Google login to a webpage resonated with me. Those complex instructions – "go to this URL, click this dropdown, select this option..." – really do feel like computers giving humans busywork. How much better would it be if agents could handle these tasks?
The Future of Developers: The Iron Man Suit Analogy
The presenter compared the future of developers to Iron Man's suit. Rather than complete AI replacement, AI becomes a tool that augments developer capabilities. The prediction is that over the next decade, this slider will move from left (pure human) to right (AI-augmented).
This is a realistic and balanced perspective. Rather than extreme scenarios of AI completely replacing developers, approaching this from a collaboration and augmentation standpoint is more reasonable.
Conclusion
This presentation offers a comprehensive and insightful perspective on software development in the AI era. The framework of categorizing software evolution into 1.0, 2.0, and 3.0 is particularly useful for understanding current changes.
This is truly an exciting time to enter the software industry. We need to write and rewrite enormous amounts of code, and both professional developers and general users can participate in this process. We must learn to collaborate with LLMs as new types of "colleagues" and build infrastructure for agents.
Most importantly, we shouldn't fear these changes but embrace them actively. Watching children naturally code while conversing with AI makes me optimistic about the future. This represents the democratization of software development and the beginning of a new era of creativity and innovation.