What's This Actually About?

Understanding the Complex: What Is AGI, Actually?

---

On November 17, 2023, OpenAI's board of directors fired Sam Altman. The stated reason was vague: he had not been "consistently candid." Within hours, Microsoft announced it would hire him. Within days, hundreds of OpenAI employees signed letters threatening to quit. Within five days, Altman was back as CEO, the board had resigned, and the company that had just been engulfed in apparent chaos was back to its mission.

What was that mission? Building artificial general intelligence — AGI — "for the benefit of all humanity."

The speed of the reversal was striking. But the more interesting question was the one that barely got asked: why did the prospect of losing control of one AI company trigger this kind of institutional panic? What, exactly, is AGI — and why do the people closest to it treat the question of who controls its development as if it matters more than almost anything else?

---

That's where this course starts. Not with a definition. Not with a timeline. With the observation that something is happening in a small cluster of laboratories in San Francisco, London, and Beijing that seems to matter enormously to the people inside it, and that most people outside have only the haziest sense of what it is.

AGI — artificial general intelligence — is a term that has been around since the 1950s and remains without an agreed-upon definition in 2026. The concept was coined in part to distinguish the broader goal of machine intelligence from the narrower project of building machines that could do specific tasks, like play chess or translate text. A calculator is not general. A chess program isn't general. Something that could do both, and also write a poem, and also design a bridge, and also notice that it was making an error in the bridge design — that gets closer to what people mean.

But "gets closer" is doing a lot of work in that sentence.

---

Here's what's concrete. There are companies that have explicitly made AGI their stated mission. OpenAI's charter describes its goal as ensuring "that artificial general intelligence benefits all of humanity," and defines AGI as "highly autonomous systems that outperform humans at most economically valuable work." Anthropic's stated mission is the "responsible development and maintenance of advanced AI for the long-term benefit of humanity." DeepMind, now Google DeepMind, describes itself as working toward "artificial general intelligence, safely and beneficially."

These aren't marketing copy, exactly. They reflect genuine convictions held by many of the people inside these labs — that what they're building toward is something genuinely different in kind from previous technology, not just better software.

They may be right. They may be wrong. They may be right about the destination and wrong about how close they are. They may be building toward something that doesn't exist in the form they imagine.

This course is about understanding that debate well enough to have a view on it.

---

A specific moment is worth sitting with. In March 2023, the AI safety organization Future of Life Institute published an open letter calling for a six-month pause on training AI systems more powerful than GPT-4. The letter was signed by over 1,000 researchers and technologists, including Elon Musk, Yoshua Bengio, and Stuart Russell — people with significant credibility in the field.

The letter wasn't calling for a pause on AI in general. It was calling for a pause on the specific category of AI development that might be leading toward AGI — or that might not be, but that nobody was sure wasn't.

Three things are notable about this moment:

First, serious people thought the stakes were high enough to publicly call for a pause. Not doom-posters on Reddit. Established researchers with tenured positions and scientific reputations.

Second, it didn't work. The labs kept training. The six months passed. No one paused.

Third: the debate it sparked — about whether we're building toward something that requires this level of caution, and who gets to decide — is still alive. If anything, it intensified.

---

The question at the center of this course is deliberately simple:

What is AGI — and why does the answer determine how we should think about AI systems today?

That question has several layers. The obvious one is definitional: what do people mean when they say AGI, and why can't anyone agree? The less obvious one is political: if AGI is achievable and close, the decisions being made right now about who builds it, how, and under what oversight structure, matter enormously. If it's not achievable or not close, the current framing of the AI debate may be generating more alarm than the evidence warrants.

Both possibilities are real. We'll examine the evidence for both.

One thing is clear: whatever AGI is, the people building the most powerful AI systems believe they're moving toward it. That belief shapes the decisions they make. Understanding that belief — its foundations, its contested nature, and its implications — is what an informed person in 2026 needs.

That's what the next ten lessons are for.

---

Next lesson: Why should I care? — Three reasons AGI matters beyond the hype and doom.

---

Reading time: approx. 8–9 minutes