Wilshire@lemmy.world to Technology@lemmy.worldEnglish · 4 months agoThe first GPT-4-class AI model anyone can download has arrived: Llama 405Barstechnica.comexternal-linkmessage-square61fedilinkarrow-up1215arrow-down117cross-posted to: [email protected]
arrow-up1198arrow-down1external-linkThe first GPT-4-class AI model anyone can download has arrived: Llama 405Barstechnica.comWilshire@lemmy.world to Technology@lemmy.worldEnglish · 4 months agomessage-square61fedilinkcross-posted to: [email protected]
minus-squaresunzu@kbin.runlinkfedilinkarrow-up3arrow-down1·4 months agoDid anyone get 70b to run locally? If so what, what hardware specs?
minus-squareDarkThoughts@fedia.iolinkfedilinkarrow-up5·4 months agoAfaik you need about 40GB of vram for a 70b model.
minus-squareDarkThoughts@fedia.iolinkfedilinkarrow-up7·4 months agoSame requirements, but much slower.
minus-squaresunzu@kbin.runlinkfedilinkarrow-up2·4 months agoI guess time to buy some ram after spending decade at 16gb
Did anyone get 70b to run locally?
If so what, what hardware specs?
Afaik you need about 40GB of vram for a 70b model.
Can’t you offload some of it to RAM?
Same requirements, but much slower.
I guess time to buy some ram after spending decade at 16gb