Calling LLMs from client-side JavaScript, converting PDFs to HTML + weeknoteshttps://simonwillison.net/2024/Sep/6/weeknotes/
-
Calling LLMs from client-side JavaScript, converting PDFs to HTML + weeknotes
https://simonwillison.net/2024/Sep/6/weeknotes/ -
@simon TIL anthropic-dangerous-direct-browser-access - Would that make it possible to access Projects and Artifacts?
-
@webology sadly I think it still only provides access to their documented API, not the fun stuff
-
Prem Kumar Aparanji ๐ถ๐ค๐replied to Simon Willison last edited by
@simon have you tried wllama?
-
Simon Willisonreplied to Prem Kumar Aparanji ๐ถ๐ค๐ last edited by
@prem_k ollama? Yeah itโs pretty good, theyโre very on top of adding new models
-
Prem Kumar Aparanji ๐ถ๐ค๐replied to Simon Willison last edited by
@simon no, wllama is the WASM binding for llama.cpp & can Inference the gguf files of the models within the browser itself. It is an alternative to Transformers.js (onnx files) and WebLLM (bin shards).
GitHub - ngxson/wllama: WebAssembly binding for llama.cpp - Enabling in-browser LLM inference
WebAssembly binding for llama.cpp - Enabling in-browser LLM inference - ngxson/wllama
GitHub (github.com)
-
Simon Willisonreplied to Prem Kumar Aparanji ๐ถ๐ค๐ last edited by
@prem_k oh fantastic! Iโve played with https://github.com/mlc-ai/web-llm but I didnโt know about the llama.cpp port, thatโs awesome
-
Prem Kumar Aparanji ๐ถ๐ค๐replied to Simon Willison last edited by
@simon #WASM is such an interesting development for web apps that can run locally in the browser, even when offline.
Slightly related note, Motherduck's 1.5 tier architecture powered by WASM is pretty cool too, especially when are able to join between tables on your browser and in your cloud in a single SQL query.
Wonder what else will WASM bring.
-
Simon Willisonreplied to Prem Kumar Aparanji ๐ถ๐ค๐ last edited by
@prem_k I love how easy WASM makes it to run tools like Tesseract - I built this OCR tool using Tesseract.js and PDF.js and it works really well https://tools.simonwillison.net/ocr