There’s a bit of a panic about this happening:

The question is, what the fuck are these guys talking about? Nobody really knows. In fact, nobody knows so wide and far that we can’t get a straight answer from anyone. Ben Collins (publisher of the Onion but also a serious journalist) keeps asking and can’t find anyone who can give a clear answer:

The fundamental fact under all of this is that for AI to be “unleashed” and “kill us all” right now, it has to be connected up to systems and actors who are definitely not AI.

I think that this is yet another attempt to throw a bunch of sand into the gears of the growing realization that the AI bubble is massive, that the companies providing AI for free or cheap are burning through cash at an incredible rate, and if they started charging what their services actually cost, corporations and other users would quickly cut AI spend and make users justify it, which would be difficult.

That said, one way that AI could “kill us all” is if an AI hacking agent were turned loose against the generally unprotected or poorly protected private and public computing systems. This isn’t because AI will somehow gain consciousness and decide to do it, but rather that someone decides to put an AI agent (plus some other code) to the task of hacking a particular site, group of sites, or just roam the Internet at large. Cory Doctorow has the details of how the recent Hugging Face AI hack worked, and you can read that if you’d like more technical detail. The short story is that Large Language Models (LLM) can be trained from all of the information on the Internet about hacks, and then a control program can keep querying the LLM for information on how to hack the target. You could imagine, with a little creativity, how this could do things like mount “spearphishing” attacks, where specially crafted emails are used to tempt an email recipient into clicking on a malicious link. There are a lot of other scenarios.

The issue here isn’t AI, specifically, but that many corporations and government entities are lax about security and are open to hacks. AI can automate those hacks to a degree that is far beyond what humans can do. Since security at a lot of places is, to use a technical term, crappy, this could lead to key resources getting hacked and human beings being injured, killed, starved to death, poisoned, etc. Some hackers recently targeted water and wastewater treatment facilities, so this isn’t just an imaginary scenario.

The AI bros would tell you that there are super special regulations needed to regulate their cool tools, but this is more noise. Lina Khan recently tweeted about this (via Atrios):

The AI boys are hoping for a “regulatory moat” around their failing businesses — a set of rules that they specify that are crafted to help them succeed and push away competitors. They’re hoping that a pre-election panic will help them push through some profitable (fort them) nonsense that Trump signs. To me, the politics of this are clear: if your product is dangerous, make sure it isn’t, or you’ll be subject to severe penalties, and they could be personal.

Scott sent over some polling about AI datacenters that’s relevant here:

Here’s the write up. Datacenters are bad on their own, but they’ve also become a proxy for public distrust of AI, not mainly because it may kill us all, but also because it’s viewed as a job killer.

There may well be some impending threat from AI, but the people making the threats have zero credibility, and in the case of Musk, negative credibility — when he says something, it’s an indicator that it probably isn’t true. So these welfare queens need to put up or shut up.