David Gerard@awful.systemsM to TechTakes@awful.systemsEnglish · 3 days agoAI computers aren’t selling because users don’t carepivot-to-ai.comexternal-linkmessage-square296linkfedilinkarrow-up1735arrow-down12file-text
arrow-up1733arrow-down1external-linkAI computers aren’t selling because users don’t carepivot-to-ai.comDavid Gerard@awful.systemsM to TechTakes@awful.systemsEnglish · 3 days agomessage-square296linkfedilinkfile-text
minus-squarewise_pancakelinkfedilinkEnglisharrow-up7arrow-down4·2 days agoLocal AI kind of sucks right now, and everything is massively over branded as AI ready these days. There aren’t a lot of compelling local use cases and the memory constraints of local mean you end up with fairly weak models. You need a high end high memory local setup to get decent token rates, and I’m finding right now 30-70b models are the minimum viable size. That doesn’t compare with speed of online models running on GPUs that cost more than luxury cars.
Local AI kind of sucks right now, and everything is massively over branded as AI ready these days.
There aren’t a lot of compelling local use cases and the memory constraints of local mean you end up with fairly weak models.
You need a high end high memory local setup to get decent token rates, and I’m finding right now 30-70b models are the minimum viable size.
That doesn’t compare with speed of online models running on GPUs that cost more than luxury cars.