Agreed with everything @kevinriggle wrote here.

Jamie McCarthy

@rgarner @jrose @inthehands @RuthMalan 100% agree with the overall thrust of what you’re saying

Paul Cantrell

That’s all well said, and gets to what @jenniferplusplus was talking about here: https://jenniferplusplus.com/losing-the-imitation-game/

Paul Cantrell

@jamiemccarthy @rgarner @jrose @RuthMalan
Yeah, I'm with Russell here: the whole “soon the AI will think” line simply isn’t justified by either theory or evidence. It’s akin to thinking that if you make cars go fast enough, eventually they’ll travel backwards in time.

Re summarization specifically…

Paul Cantrell

@jamiemccarthy @rgarner @jrose @RuthMalan
…there was a recent paper (lost link, sorry) that systematically reviewed LLM-generated summaries. They found in the lab what people have observed anecdotally: LLMs suck at it because they don’t know what the point is. They’re great at reducing word count in a grammatically well-formed way! But they often miss the key finding, highlight the wrong thing, etc.

Paul Cantrell

@muhanga @gregdosh @sethrichards
Another version of this I've heard is that AI-generated code reduces the “bus factor” of your team to zero the moment the code is written.

Shafik Yaghmour

@inthehands @kevinriggle

What I am seeing from studies is pretty troubling e.g. https://hachyderm.io/@shafik/113391494588227091

Folks feel more productive but actual objective measures say otherwise.

It is never about lines of code produced that is a silly measure by itself.

It is really about how many errors make it into production. If using AI code generators means a lot more bad code makes into production you are making a very poor tradeoff.

Old Fucking Punk

@inthehands My 30+ years as a dev disagree with the statement, "coding a solution is a deep way of understanding a problem", on semantics. Analyzing a solution is a deep way of understanding a problem. I've found that too many devs are in a rush to code and give short shrift to the analysis.

Paul Cantrell

@lwriemen This is an overly pedantic quibble, though I agree with the underlying sentiment that people rush into coding too fast without thinking.

Doing the work of filling the things I left between the lines for the reader to infer above: coding something •while thinking• — assessing the results, letting the ideas talk back and surprise, treating design and implementation problems as prompts to think about goals and context — is a deep way of understanding a problem.

Sébastien Roccaserra 🐿️

@inthehands @dymaxion @kevinriggle @jenniferplusplus This is interesting. I have another concert: what if the goal of the GenAI narrative was mostly an accounting goal? Meaning, tech is / was worth investing in because the huge promise of future growth enables to boost the balance sheet & enables to pay nice dividends every year. So you have to regularly renew the narratives that justify promises of future growth. GenAI is just that, it doesn't even *need* to be a real productivity boost.

Cassandrich

@sroccaserra @inthehands @dymaxion @kevinriggle @jenniferplusplus This is absolutely what it is and what we've been telling folks it is for years. Of course they won't listen. See: my display name.

Paul Cantrell

@dalias @sroccaserra @dymaxion @kevinriggle @jenniferplusplus

A shockingly large portion of the world’s current problems trace back to an excess of investment money desperate for more things to invest in.