Calling LLMs from client-side JavaScript, converting PDFs to HTML + weeknoteshttps://simonwillison.net/2024/Sep/6/weeknotes/
-
Calling LLMs from client-side JavaScript, converting PDFs to HTML + weeknotes
https://simonwillison.net/2024/Sep/6/weeknotes/ -
@simon TIL anthropic-dangerous-direct-browser-access - Would that make it possible to access Projects and Artifacts?
-
@webology sadly I think it still only provides access to their documented API, not the fun stuff
-
Prem Kumar Aparanji πΆπ€πreplied to Simon Willison last edited by
@simon have you tried wllama?
-
Simon Willisonreplied to Prem Kumar Aparanji πΆπ€π last edited by
@prem_k ollama? Yeah itβs pretty good, theyβre very on top of adding new models
-
Prem Kumar Aparanji πΆπ€πreplied to Simon Willison last edited by
@simon no, wllama is the WASM binding for llama.cpp & can Inference the gguf files of the models within the browser itself. It is an alternative to Transformers.js (onnx files) and WebLLM (bin shards).
GitHub - ngxson/wllama: WebAssembly binding for llama.cpp - Enabling in-browser LLM inference
WebAssembly binding for llama.cpp - Enabling in-browser LLM inference - ngxson/wllama
GitHub (github.com)
-
Simon Willisonreplied to Prem Kumar Aparanji πΆπ€π last edited by
@prem_k oh fantastic! Iβve played with https://github.com/mlc-ai/web-llm but I didnβt know about the llama.cpp port, thatβs awesome
-
Prem Kumar Aparanji πΆπ€πreplied to Simon Willison last edited by
@simon #WASM is such an interesting development for web apps that can run locally in the browser, even when offline.
Slightly related note, Motherduck's 1.5 tier architecture powered by WASM is pretty cool too, especially when are able to join between tables on your browser and in your cloud in a single SQL query.
Wonder what else will WASM bring.
-
Simon Willisonreplied to Prem Kumar Aparanji πΆπ€π last edited by
@prem_k I love how easy WASM makes it to run tools like Tesseract - I built this OCR tool using Tesseract.js and PDF.js and it works really well https://tools.simonwillison.net/ocr