I once worked at a company that sold industry-specific core-business software to deep-pocketed corps who couldn’t / wouldn’t / shouldn’t roll their own.

Wolf480pl

@inthehands in a sane world, after you realized the whole health score was a hoax, you'd be able (or even required) to report them to some kind of institution that would send inspections and lawsuits their way.

We don't live in a sane world.

Wolf480pl

@inthehands Also, regarding the customer support example:
Yes, it saves time for the company, and it was valuable human time that was being spent on pointless things.

But it makes the *customer* waste *more time* doing pointless things

Moreover, it creates an assymmetry, where the company can spend relatively little time to make the customer waste a lot of time.

So AI is a weapon.

And as such, it should be regulated.

Chris Ammerman

@inthehands This thread started out as incredibly deflating and ended up flatly horrifying.

Jeremy D. Miller

@inthehands You're giving me so many flashbacks -- and not the good time -- to being an internal architect in "Big IT" in the early 00's

Paul Cantrell

@Npars01
Yep, this is all at the heart of it. It’s the same thing that brought us the 2008 crash: too much investor money looking for returns that don’t exist. Then it was just investor wanting more mortgages to invest in than there were mortgages to reasonably offer. But this time, it’s broader, and I fear much worse.

George Ellenburg (he/him/his)

@[email protected] I'm calling it. You either worked for Oracle or IBM.

Exandra

@inthehands thanks Paul, your posts helped set to rest some confusion I had about gen AI: how can people, knowing it is bad for their own businesses when it inevitably and continually fails and implicates them, still want to incorporate it into their products.

Your thread answers this question!

Paul Cantrell

@froge @zannesan
I mean, rich fucks, yes, but: it’s systemic. Large-scales forces create and reward their existence. If these people went away or became better, others would appear to take their place.

Paul Cantrell

@rrdot
Cheers! Screaming into the void may be futile, but I guess it’s nice when somebody enjoys the concert?!

Paul Cantrell

@thias
Absolutely. Getting to the place where Goohart is the relevant problem was my foolish dream.

Paul Cantrell

@peter
Yeah, the first example is the only one in the thread that's •not• bullshit. Point is that (1) if you say it’s terrible, you have to ask “As compared to what?,” and point of the later examples is (2) the true business goal isn’t always what you think it is.

If we want to understand the function of LLMs for corps, we have to work through those questions. And given the amount of work that’s •already• BS, I’m not necessarily shorting the LLMs even though they’re BS too.

Advanced Persistent Teapot

@kaasbaas @inthehands like the well being of staff for example

Paul Cantrell

@stew_sims
Yeah. For example: some of those ancient mainframe systems are just rock solid: somebody built it right in like 1972 in COBOL or whatever, and it’s far wiser to do hardware maint to keep the same code running than to attempt a rewrite!

Point is that in my first example, yes, the bad product really •was• better. “Compared to what?” and “Toward what goal?” are both questions that can have very surprising answers.

Paul Cantrell

@sennoma
sorry

Paul Cantrell

@plsik
sorry/welcome?

Paul Cantrell

@KatS
You’re agressively agreeing with me: the point is that, in that first example in my thread that you're replying to, our terrible product really •was• an improvement. Nobody was being swindled. The companies knew how bad our stuff was, and knew how much worse their current processes were. No hoax there.

—

Students are always astonished when I tell them that of course you release software w/bugs. “Ideally, a good team understands •what• bugs they're releasing.”

Paul Cantrell

@Stevenheywood
Back then and still today, I think

Paul Cantrell

@paulmwatson
I mean, yes, and to hell with the church of Altman. I do think this is a bubble and there will be a crash.

Paul Cantrell

@penguin42
Yup, definitely recognize “nobody’s actually gotten that far” from support — and I’ve been on both sides of that conversation!

Paul Cantrell

@wolf480pl
Someone once remarked that patients are the free labor of the US medical system, and that’s really stuck with me. And it’s not just medical where that pattern occurs.