Flying Squid@lemmy.world to Technology@lemmy.worldEnglish · 9 months agoYour AI Girlfriend Is a Data-Harvesting Horror Showgizmodo.comexternal-linkmessage-square97fedilinkarrow-up1494arrow-down113cross-posted to: [email protected][email protected]
arrow-up1481arrow-down1external-linkYour AI Girlfriend Is a Data-Harvesting Horror Showgizmodo.comFlying Squid@lemmy.world to Technology@lemmy.worldEnglish · 9 months agomessage-square97fedilinkcross-posted to: [email protected][email protected]
minus-squareexu@feditown.comlinkfedilinkEnglisharrow-up4·9 months agoYou’ll want to use a quantised model on your GPU. You could also use the CPU and offload some parts to the GPU with llama.cpp (an option in oobabooga). Llama.cpp models are in the GGUF format.
You’ll want to use a quantised model on your GPU. You could also use the CPU and offload some parts to the GPU with llama.cpp (an option in oobabooga). Llama.cpp models are in the GGUF format.