Here's a neat trick for libraries that are under-documented but have comprehensive tests: use the CLI combo of llm and files-to-prompt to generate your own private documentation directly from their test suite! https://til.simonwillison.net/llms/docs-from-tests
Full transcript here: https://gist.github.com/simonw/351cffbd254af5cbf329377fb95fcc13
@xsc for personal stuff absolutely - and yes for most of the things I'm building on the APIs
@xsc it has a rate limited free tier, as do the other Gemini models
OpenAI reduced a slightly confusing feature today called "Predicted Outputs", which lets you spend more money on a prompt to have it return faster if the output is mostly predictable based on the input (e.g. a prompt that makes edits to some code) - detailed notes here: https://simonwillison.net/2024/Nov/4/predicted-outputs/
Wrote more about Claude 3.5 Haiku on my blog, including this table showing how the price compares to other leading hosted models https://simonwillison.net/2024/Nov/4/haiku/
I released a new version of llm-claude-3 adding support for the new model (and fixing an attachments bug): https://github.com/simonw/llm-claude-3/releases/tag/0.8
llm install --upgrade llm-claude-3
llm keys set claude
# paste API key here
llm -m claude-3.5-haiku 'impress me with your wit'
I also added 3.5 Haiku to my LLM pricing calculator: https://tools.simonwillison.net/llm-prices
Claude 3.5 Haiku is out, with a couple of surprises:
1. It's priced differently from Claude 3 Haiku. 3.5 Sonnet had the same price as 3 Sonnet, but 3.5 Haiku costs ~4x more than 3 Haiku did
2. No image input support yet
3.5 Haiku supposedly beats 3 Opus though, and Opus cost 15x the new Haiku price!
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
(www.anthropic.com)
@tuananh doesn't help much if the user can do anything that you don't want the LLM to also be able to do
@daniel yeah it's really good - Claude 3.5 Sonnet has been my daily driver since they release it
Best model I've used for code too
Added a section helping explain what's going on with our dog Cleo: https://gist.github.com/simonw/d680d5f27b50de4efc26ed2a673292fd
@matt I tried it on two documents so far and it looked reasonable, but I've not done a remotely robust comparison of it yet
@xsc I tried it on two PDDs and it looked OK, which isn't nearly enough testing for me to say anything useful!
@orangeacme doesn't tell me if I get an hour of extra sleep or not though!
I always have to think hard about what "clocks go back" actually means
Because I can never figure out what's going to happen with Daylight Savings Time changes I got Claude to build me my perfect California DST app https://tools.simonwillison.net/california-clock-change
Claude transcript here https://gist.github.com/simonw/9510723176f5b44ac1ebc495c95a4bc7
Published some notes on Docling, a rather nice MIT licensed Python PDF document / table extraction library from IBM https://simonwillison.net/2024/Nov/3/docling/
@ericholscher @webology @rochecompaan yeah, one of the most exciting trends right now is that the capable models keep getting smaller and easier to run - I've been having fun with this one: https://simonwillison.net/2024/Nov/2/smollm2/
@webology @ericholscher listen to Jeff, he runs more local models than I do!
@ame let me know if it works, I'm not sure how best to test it!
@ame I got Claude to make an artifact for that instead, looks like it might work OK https://gistpreview.github.io/?e29943852f371f638c9a3ae1dcc4784e
Claude transcript: https://gist.github.com/simonw/49b52ce2a7b5796edf4e0e2e2152db41
@ericholscher optimise for RAM, that's the limiting factor on how good a model you can run
I filled uo my 1TB hard drive with models pretty quickly, but loading them off an external drive seems to work fine if you have the memory for them
