I’m not sure that’s even a valid comparison? I’d love to know where you got that data point.
LLMs run until they decide to output an end-of-text token. So the amount of power used will vary massively depending on the prompt.
Search results on the other hand run nearly instantaneously, and can cache huge amounts of data between requests, unlike LLMs where they need to run every request individually.
I’d estimate responding to a typical ChatGPT query uses at least 100x the power of a single Google search, based on my knowledge of databases and running LLMs at home.
E: I’ve cleaned up the comment for saucing ease in case you looked at it right away but I’ll quote it here for ease.
Yep, I’m familiar with it. More from previous comments since these disingenuous memes get pedaled here regularly. People love to spout stats about AI in data centers that aren’t just used for AI without having any sense of how much CO2 is produced from really really common stuff lol. Let alone it being contingent on being run in non-renewable powered areas yet 40% of the USA’s grid is clean.
How about watching TikTok? 30 minutes of video daily = 28kg of carbon a year. Calculators here: 1, 2, 3.
How about the recent ‘Oh no 1/5 a city’s water!’ The city use 770,000 gallons a day… So it uses 154,000 gallons of water for an entire piece of critical infrastructure that keeps the internet running. For the entire data center lol. And that’s for the city Carrol, Iowa with a whopping population of 10,000 people in 5 square miles lol.
A real common citation is how much carbon it takes to initially train these too. 500 tons of carbon dioxide… That’s only 33/334,000,000 Americans worth of CO2 for the year lol.
0.3Wh / request for Google
2.9Wh / request for ChatGPT
That does however reference the same paper as your linked articles, which I can’t find without a paywall: https://www.sciencedirect.com/science/article/abs/pii/S2542435123003653?dgcid=author
I’d love to know how they came up with that number for ChatGPT, but it looks like I was a bit off with my estimates regardless. There’s probably some scaling efficiencies they’re taking advantage of at that size.
I’m not sure that’s even a valid comparison? I’d love to know where you got that data point.
LLMs run until they decide to output an end-of-text token. So the amount of power used will vary massively depending on the prompt.
Search results on the other hand run nearly instantaneously, and can cache huge amounts of data between requests, unlike LLMs where they need to run every request individually.
I’d estimate responding to a typical ChatGPT query uses at least 100x the power of a single Google search, based on my knowledge of databases and running LLMs at home.
https://old.lemmy.world/comment/10803727
E: I’ve cleaned up the comment for saucing ease in case you looked at it right away but I’ll quote it here for ease.
Thanks for the links. I was able to find the original source for that claim, which has actually usage numbers: https://iea.blob.core.windows.net/assets/18f3ed24-4b26-4c83-a3d2-8a1be51c8cc8/Electricity2024-Analysisandforecastto2026.pdf
0.3Wh / request for Google 2.9Wh / request for ChatGPT
That does however reference the same paper as your linked articles, which I can’t find without a paywall: https://www.sciencedirect.com/science/article/abs/pii/S2542435123003653?dgcid=author
I’d love to know how they came up with that number for ChatGPT, but it looks like I was a bit off with my estimates regardless. There’s probably some scaling efficiencies they’re taking advantage of at that size.