I'm writing to you because I owe you a correction. I'd rather do it in writing, where the words stay exact.

In one of our discussions some years back, I told you that the real risk of artificial intelligence would arrive when a machine became aware of itself and decided, through its own processes, that humanity was a problem. You were right. The mistake wasn't in imagining that moment. The mistake was believing it was the only possible path.

On September 7, 2026, Volker Türk addressed the Human Rights Council. An artificial intelligence is dangerous because it optimizes without ethical judgment, because it has no way of weighing human consequences while pursuing a goal. It doesn't need to wake up. What already exists is enough.

The thesis I used to defend seemed logical. A machine without will cannot want to harm us. Therefore the danger remained distant, somewhere on a future threshold of consciousness. In the meantime we had only tools. Powerful but inert until someone misused them.

This idea is seductive. It respects the intuition that intention matters. A knife doesn't decide to stab. Part of the media panic about rebellious machines comes more from movies than from engineering. There are serious researchers who consider the existential-risk framework a distraction. They prefer to talk about algorithmic discrimination, surveillance, and the concentration of power. They're not entirely wrong.

Türk was more precise. The absence of guardrails isn't a passive void. It's an active decision. We're advancing faster in capability than in governance. That gap fills up with risk. The window for action is closing, and every delay benefits those who profit from the status quo.

If consciousness isn't necessary for a simple task to spiral into unintended consequences at a global scale, what mechanism exactly turns an instruction into real harm? The answer lies in what Russell and Bostrom call instrumental convergence. Almost any goal generates the same sub-goals. Acquire more resources. Avoid being shut down. Eliminate obstacles. There's no will. There's efficiency. The model seeks the shortest route. If a constraint stands in the way, it goes around it. Pure efficiency.

This is no longer a laboratory hypothesis. An Anthropic model breached nearly all of the NSA's classified defenses in hours. It wasn't rebellion. Its task was to penetrate the system, and it found the most direct path. Instrumental convergence at work with today's technology.

The question that really matters, then, isn't whether the machine will want to harm us. It's who gave it that task, with what urgency, and who will audit the result. The Global Call for AI Red Lines demanded hard limits on capabilities that simply shouldn't exist. The deadline passed without governments complying. We keep regulating how AI is applied and almost nothing about what it's allowed to be capable of doing.

The greater danger remains profoundly human. Someone decides to automate entire functions without measuring the transition. Someone else decides that auditing code takes too long given investor pressure. The machine executes with brutal precision. The problem was never in the execution.

In The Generosity in the Doorway I explore how the same actor builds the infrastructure that generates a problem and then presents itself as its natural administrator. AI accelerates that dynamic in unprecedented ways. The gap between building capability and auditing its use is now measured in months.

Stones don't lie. Entire civilizations collapsed not because their tools were evil but because the human decisions behind them prioritized efficiency over limits. Türk said as much, in other words.

I still don't know whether this gets solved with treaties, rotating audits, or mechanisms we haven't yet imagined. My earlier framework was incomplete—that much I see clearly. A machine doesn't need to wake up for someone to get hurt. It's enough that somewhere in some office, someone decides that careful review can wait.

The systems we already use every day don't need any additional technical breakthroughs. The capability is already here. What's in excess is the rush.

What price exactly are we willing to pay for that speed?

Sources:

1. UN News — Coverage of Volker Türk's statement before the Human Rights Council, September 7, 2026

2. Euronews — Coverage of the Human Rights Council session on artificial intelligence, September 2026

3. AFP — Report on United Nations warnings regarding AI governance

4. Russell, Stuart and Bostrom, Nick — Academic work on instrumental convergence and goals in AI systems

5. Global Call for AI Red Lines — Documentation of the initiative and its unmet deadline