The environmental comparison I'd be interested in seeing is between a year of heavy personal usage of LLMs (ChatGPT, Claude, Gemini etc) compared to the CO2 emissions from a single passenger flight
-
@simon wait what, you have air trips every year? Let alone MULTIPLE air trips? Ok, sure, financially supporting LLM training and using cloud LLMs is probably not a significant addition to *your* resource consumption...
-
@scott the problem I have with that model is that, taken to its logical conclusion, I shouldn't travel by car or bus or train, or purchase manufactured goods, or turn on the heating, or - you know - live
-
Simon Willisonreplied to Simon Willison last edited by [email protected]
@scott if I'm going to do stuff that emits carbon I need to have some kind of framework for deciding what things I do and what things I don't
-
@kitten_tech having family in the UK having moved to the US does lead to quite a lot of additional CO2!
-
@simon interesting question.
How would you account for the energy spent on training all these models? That costs much more energy than the inference, but it is also spilt on everyone using the models...
-
Simon Willisonreplied to Ruben last edited by [email protected]
@ruben_int this leaked chart suggests OpenAI spent 2bn to run models and 3bn to train them, so a 2.5x multiplier might be a good way to estimate running + training costs
-
@simon no
-
@jfroehlich why not?
-
@simon if you would like to see such a comparison, why not rephrase your request as a prompt and feed it to some of your favorite LLMs?
-
@trindflo I already did exactly that - didn't even rephrase it, just pasted it in
-
Simon Willisonreplied to Simon Willison last edited by [email protected]
@trindflo here's what I got from ChatGPT, problem is I don't trust ANY of the numbers because the underlying information on how much energy an individual prompt uses is effectively unavailable
Throwing in "What are some ethical and philosophical framework I could use to help explore this question?" did bring up some interesting leads though
https://chatgpt.com/share/6712f5dd-9bdc-8006-9a0a-f352ccb5b7f3
-
@simon I've found the self-reported Llama2 training (!) emissions interesting. See section 2.2.1 here:
Llama 2: Open Foundation and Fine-Tuned Chat Models
Abstract page for arXiv paper 2307.09288: Llama 2: Open Foundation and Fine-Tuned Chat Models
arXiv.org (arxiv.org)
That doesn't say much about inference, tho, but it's a start.
You'd also have to include not just GPU energy but e.g. emissions of the data center and supporting software around it, cluster utilization, the emissions of people working on this, traveling, and so on.
There's Scope1/2/3 etc. categories to make this a bit more structured but in the end it's tough.
-
@simon If you want to balance out your own emissions (if that even makes sense, see folloing toot), you need to make sure your efforts adher to a concept from the climate space called "additionality":
Your skipped flight only counts if you really would have taken it and you cancel it not because you can but because you have to.
-
@djh thanks for "additionality" - that's exactly the kind of terminology I was hoping to learn about today
-
@simon yes, if it is at all afforded to you by your society, you should avoid doing things that consume energy up until the point where we've balanced our energy use budget with the earths resources. If you do not personally have that ability, it's very understandable: collective energy use requires collective efforts to reduce usage. Change must be pressed for at the societal level.
That said, it is absurd and ridiculous to compare the necessity to heat your home to the vanity of consuming that power in the form of a fancy lying-to-you machine is so ridiculous I can't believe I actually have to say it. I have never used one of these products and I'm still alive and well.
-
@scott "I have never used one of these products and I'm still alive and well"
Then it's understandable that you wouldn't see why I value them more than an international flight!
I use them for a lot of things, some of it dumb but most of it genuinely useful https://simonwillison.net/series/using-llms/
-
@simon your writing often encourages others to use LLMs more. Would you like to take personal responsibility for your influence on other people's emissions? Have I framed the question fairly?
-
@graham_knapp I do think about that - am I having an outsized negative effect by showing people how to use this stuff?
I figure people are going to use this stuff anyway, so I hope to have a positive impact by encouraging people to use it responsibly and productively (and not just to generate slop)
