Do you ever use LLM tools like Claude or ChatGPT to help code up exploratory prototypes?
-
Do you ever use LLM tools like Claude or ChatGPT to help code up exploratory prototypes?
-
@simon I'm surprised by the % saying they disagree with the ethics when *prototyping*; thats interesting.
-
@ijm my guess is that a lot of that could be about the energy usage and environmental impact, as illustrated by this recent thread https://toot.cafe/@baldur/113017028974375684
-
Simon Willisonreplied to Simon Willison last edited by
@ijm and to be honest, “it’s just for the prototype” as an ethical workaround doesn’t hold up when if the prototype is any good the chances are quite a bit of it will end up used in the final product
-
@simon For me I find a lot of the value of spikes and prototypes comes from the process, e.g. discovering that things work differently than I expected. Even if an LLM can give me a working prototype I am worried about the loss of that learning and discovery. I might be wrong though. I'll try it next time and see.
-
@twp I'm finding that LLM prototypes are accelerating that process for me too - it's much quicker to try a different approach ("what if I do this with a subprocess instead of threads, how about if I use a SQL UNION here, could AppleScript get this done for me better?") and I'm still reading the code so I'm still learning from what works and what doesn't.
-
@simon
In my brief exploration of it (and based on others experience) it seems to be a direct replacement of Stack Overflow.That is, if you use reasonably mainstream technology, and you want help in solving a common problem or implementing a standard solution, perhaps with a small twist, then it's helpful and generally correct.
But as you veer off the mainstream path, the suggestions rapidly become misleading and wrong, and it's faster figuring it out for yourself.
-
@jannem I’ve not been finding that myself - sure, it’s best at Python and JavaScript and SQL but I’ve been getting great results for languages I don’t know well (or at all) like Go and AppleScript
You gotta get good at testing what it proxies, but that’s a similar skill to code reviewing code by other people
-
@simon
I was trying to get help on Chapel, but that failed pretty badly.And if I try with Gdscript the models will tend to give me Python code instead. It looks quite similar, and with orders of magnitude more training examples that's perhaps not unexpected.
Oh, and I did try getting help on writing lock-free parallel code in Julia but that failure may honestly be a Julia problem more than the models'.
-
@jannem hah, yeah your choice of programming languages is a whole lot less mainstream than mine!