Some_Emo_Chick@lemmy.world to Technology@lemmy.worldEnglish · 3 months agoGenerative AI Is an Engineering Disaster. A shockingly inefficient trillion-dollar project.www.theatlantic.comexternal-linkmessage-square193linkfedilinkarrow-up11arrow-down10
arrow-up11arrow-down1external-linkGenerative AI Is an Engineering Disaster. A shockingly inefficient trillion-dollar project.www.theatlantic.comSome_Emo_Chick@lemmy.world to Technology@lemmy.worldEnglish · 3 months agomessage-square193linkfedilink
minus-squareBrett@programming.devlinkfedilinkEnglisharrow-up0·2 months agoWhat are you using to run the model? Llama.cpp will automatically split the model between your system ram and graphics card’s vram. Qwen 3.6 is a mixture of experts model with only 3B parameters active at a time. Even without quantization your card could easily run that.
What are you using to run the model? Llama.cpp will automatically split the model between your system ram and graphics card’s vram.
Qwen 3.6 is a mixture of experts model with only 3B parameters active at a time. Even without quantization your card could easily run that.