Anthropic released the system prompt for their various consumer LLM chatbot apps today, and they're a really fun read. Made some notes on them here: https://simonwillison.net/2024/Aug/26/anthropic-system-prompts/
-
@simon Yeah, no kidding. Any sign of a customizable base user prompt a la ChatGPT?
And, follow-up question, assuming you're using that feature of ChatGPT, how helpful do you think it has been?
-
@michaelgemar that will never cease to amuse me!
-
@judell you can get a bit of that with Claude "Projects" - I have one which thinks it's a manatee who's an expert at vegetarian recipes and salads
I don't use it for ChatGPT because I don't like the idea that my interactions with the models might be biased in unexpected ways by my custom instructions - I want to gain as much experience as possible with the base model that everyone else is using
-
Simon Willisonreplied to Simon Willison last edited by
@judell here's my manatee reverse engineering a photo of some avocado toast for me https://gist.github.com/simonw/7ae554005d788ca233440f18d067e4c6
-
@simon it was also shared on /r/claudeai where a substantial portion of the community are convinced sonnet 3.5 has degraded significantly in recent weeks. https://old.reddit.com/r/ClaudeAI/comments/1f1shun/new_section_on_our_docs_for_system_prompt_changes/
Per the rep "We've also heard feedback that some users are finding Claude's responses are less helpful than usual. Our initial investigation does not show any widespread issues. We'd also like to confirm that we've made no changes to the 3.5 Sonnet model or inference pipeline."
-
@simon It’s darkly ironic, but it also indicates how much of a black box the trained models are even to the companies creating them, which isn’t great.
-
Simon Willisonreplied to Alex Bradbury last edited by
@asb yeah the superstition about the models getting weaker is always so interesting to me - especially when the LLM reps say things like "we haven't deployed any changes, it's the same exact binary on disk" - here's another of those from April https://simonwillison.net/2024/Apr/15/jason-d-clinton/
-
Simon Willisonreplied to Michael Gemar last edited by
@michaelgemar yeah, we're several years into the LLM revolution now and still nobody fully understands how they work!
-
Quinn Comendantreplied to Simon Willison last edited by
@simon I haven’t had much success using negative prompting. I’m surprised they get this to work.
-
Simon Willisonreplied to Quinn Comendant last edited by
@com yeah same here, telling models not to do things isn’t that effective in my experience
I’m pretty sure I’ve still seen Claude say “certainly”!