OpenAI’s Whisper audio transcriber makes up hospital treatments and medications
-
OpenAI’s Whisper audio transcriber makes up hospital treatments and medications
OpenAI’s Whisper audio transcriber makes up hospital treatments and medications
Machine learning for audio transcription is not perfect — you always have to check the output — but it’s often good enough in practice for real work. But OpenAI sells LLMs. When you sell hammers, e…
Pivot to AI (pivot-to-ai.com)
-
@davidgerard where does it say OpenAI run whisper transcriptions through ChatGPT/GPT-4? I couldn't find that bit
-
Simon Willisonreplied to Simon Willison last edited by
@davidgerard my experienced of Whisper hallucinations so far has been the less exciting variety where things like blank space are turned into "Thank you for watching!"
-
@simon I'm presuming that from the hallucinations, which sound extremely LLM. Pure ML doesn't make up stuff like that. Also, OpenAI is very one-trick.
-
Simon Willisonreplied to David Gerard last edited by [email protected]
@davidgerard I'm pretty confident those were pure Whisper hallucinations, not anything to do with running through another LLM
Whisper has the capacity to hallucinate purely from its own model - similarly, it can translate from one language to another (e.g. Spanish to English) entirely in-model, which is a good hint as to how complex the model is under the hood
-
Simon Willisonreplied to Simon Willison last edited by
@davidgerard from digging around in the data from that paper - https://github.com/koenecke/hallucination_harms - it looks to me like the underlying story here is that Whisper v2 hallucinated wildly but Whisper v3 (released November 2023) was a huge improvement
