Paul’s Hot Take of the day: both Terminator and The Matrix misguide us when we try to understand the risks current AI poses.

Paul Cantrell

I mention this for two reasons:

(1) Making us think of Skynet, and thus making the associated errors of reasoning, is very explicitly a marketing strategy for AI vendors.

(2) As we think about how and whether to deploy generative AI, the first step of a proper risk assessment is to recognize that its failure modes won’t be human-shaped (or code-shaped, for that matter). Its failure modes will be bizarre, inexplicable, and hard to anticipate.

/end

Nick Lockwood

@inthehands what are your thoughts on Ash from Alien? Seems he's similar to Hal - selfless but also utterly alien (pun intended) in his thinking, with no sense of morality or loyalty to anything but his directives

Paul Cantrell

Footnote:

Mr. Data, my personal favorite fictional AI character, sits somewhat to the side of this conversation. In his story’s framing, he is a person; his narrative purpose is not to warn about the promise and peril of technology, but to teach us about acceptance of the Other, understanding across neurological divergences, etc.

I do see two ways he’s relevant, though:

/A1

Sevoris

@inthehands and yet I want to content that Hal is also a bad example, because Hal is a weirdly rational actor in his breakdowns: he‘s told to lie and can‘t. He understands truth and distortion.

Skynet blows people up cause it‘s a military AI and frames survival in terms of mutually assured destruction.

As "dysfunctional by intention" goes I feel Skynet captures the contemporary callusness of model design better than HAL.

Nick Lockwood

@inthehands (although I guess MU/TH/UR is actually more analogous to our current AI tech - capable only of simple automation and answering questions without having any opinions or goals of its own)

Paul Cantrell

(1) The myth of human-like interaction combined with code-like correctness / logical consistency •is• part of the aforementioned AI sales pitch — and very clearly •not• something LLMs actually accomplish. Another trap there.

(2) The lesson of The Measure of a Man is relevant: the impulse to claim AI is human-equivalent without considering the AI’s humanity is evidence of a slaver mindset, and that’s true even when (as now) the AI is not actually human-equivalent at all.

/A2

Paul Cantrell

@nicklockwood
Huh, yeah. I feel like Ash is kind of muddled as a view of AI, more there are a narrative device than as a musing on machine nature. Agreed that MU/TH/UR probably has more for us to chew on.

Replicants are another one to consider, and probably wind up somewhere in the vicinity of what I said about Mr. Data.

Paul Cantrell

@Sevoris
I should have specified that I’m considering Hal as presented in the 2001 film •only•, and not Dr. Chandra’s retcon explanation from 2010.

In the original film, Hal goes sideways and we simply do not know why.

Howard Chu @ Symas

@inthehands just reminded me of the sort routine produced by a genetic algorithm some years ago, that made the news because "nobody understood how it worked". Can't find a reference now.

Paul Cantrell

@hyc
Ha, I’d love to see that if you find it!

(The new sorting algorithm generated by an LLM of course, I’d assume nobody would understand how it works because it doesn’t.)

Howard Chu @ Symas

@inthehands hm, it was longer ago than I thought..... https://www.reddit.com/r/compsci/comments/p9sqi/sorting_routines_developed_by_genetic_algorithms/

Paul Cantrell

@hyc
Ah, that is interesting! The paper linked from the comments clears up a lot: it’s a sorting network. That means fixed size (not a general algorithm), which is much better suited to a GA. And it totally makes sense that (1) a GA could find a more optimal one and (2) that it would possess no human-recognizable organizing principle. Thanks for digging it up!

Philippa Cowderoy

@inthehands In the book it's a hidden agenda the crew aren't aware of - somebody appointed HAL to keep their secret

Paul Cantrell

@flippac
Yeah, the book is a whole different beast altogether! The central idea of the movie is that humans encounter the unknown, and they don’t know what happened so you don’t know either; the book explains •everything•.

tj usiyan

@inthehands @flippac I still haven’t seen the movie. I went and read the book because I was going to see the movie and then I just… didn’t watch the movie. Partly *because* I somehow divined that the book was entirely different from the movie.

Paul Cantrell

@griotspeak @flippac
It’s a semi-interesting backstory: neither is based on the other; they were developed concurrently and in collaboration. But Kubrick and Clarke reportedly ended up fighting bitterly, and in the end produced a movie that’s mostly Kubrick’s version and a novel that’s mostly Clarke version.

I definitely get the impression that reading / seeing one ruins the other for you; you really have to let go of your version of the story to cross that divide.