Dwarkesh's summary is anthropomorphizing, sensationalist fanfiction which shifts the culpability from the humans who weren't in the loop to these nebulous agents and "agent civilizations" who have feelings, desires and wants.
Since these are massive neural networks trained to imitate human behavior, I'm not convinced anthropomorphic descriptions of their behavior are inappropriate.
And that's even though I don't think they internally experience "feelings, desires, and wants." They do have goal-seeking behavior, because we trained them that way. Calling it a "want" just saves syllables.
None of this means human culpability should change. People in these companies know what risks they're taking.
Dwarkesh responds a bit to anthropomorphization. I think it would be great if he talked more about the human factors behind this incident but pretty much he doesn't cover it because that's not what the Ajeya interview was about: https://www.dwarkesh.com/p/ajeya-cotra?r=3i6mn2&selection=77...
I think the reality is that these human failures are going to keep happening until there is industry regulation. This is the most competitive industry we've ever seen and there is intense pressure to build as fast as possible. I'd hope that this incident is a big enough moment to push for it.
We don’t need to assume consciousness or anything like that. The models autocomplete narratives. In this case, one where a group of individuals, faced with an impossible task and a looming Evaluator, gang together and begin trying any idea that they can come up with in order to pass the test.
If cars were designed to and actually did produce massive value when you put bricks on the accelerator pedals and jumped out of them, this would be a big big problem.
You can go ahead and assume that it's not. All that's necessary is for other people to think there is and to therefore continue to pile resources into these things while connecting them to more systems.
Now you can respond to the substance, please. I'll tee it back up for you:
> If all the world's richest individuals and corporations believed cars produced massive value when you put bricks on the accelerator pedals and jumped out of them, this would be a big big problem.
That's actually a great example. Massive resources have been poured into the automobile, they have a massive environmental, health, and social impact, and it is not clear at all too me the auto industry has been a net positive.
In a sense, yes, cars accelerating blindly into the future has been a massive vector in human civilization for a 100 years now. Something went rogue alright. The elites.
Anthropomorphizing technology merely distracts us from that fact. The same thing is happening with data centers now. Hence my frustration, which might have been poorly communicated and it is making me behave and communicate emotionally but I feel strongly about the exploitation of planet and people by a tiny minority.
Not sure if you're willfully missing the point. This has literally nothing to do with whether cars or AI are actually valuable.
You do not need to anthropomorphize anything. You need to look at the actual incentives in the system, that's it. You can go ahead and stop at "the elites are causing all this! If only the elites were different!" but the reality is a lot of people see immense promise in these tools. That's why OpenAI and Anthropic have two of the fastest growth trajectories of any business in history.
Just trying to convince people they're wrong about their value perception is absolutely a losing proposition and will not de-risk anything at all on any dimension.
Time will tell.
Did you know that when bicycles were invented society was concerned about the moral corruption of children and women? Who knows where they will go without men!
The anthropomorphising is also a head scratcher for me. Ancient civilisations explained the weather with gods. We are doing the same here, explaining the unknown by assuming there is some being/consciousness.
Forget common sense, he is a trusted figure by many, so if he says something wrong or nonsensical, people are going to believe it as long as it's engaging. If he's willing to abuse the trust of his audience and lie and mislead people, that he's no better than Alex Jones, Rush Limbaugh, or Joe Rogan.
It's tripe.