Some_Emo_Chick@lemmy.world to Technology@lemmy.worldEnglish · 28 days agoGenerative AI Is an Engineering Disaster. A shockingly inefficient trillion-dollar project.www.theatlantic.comexternal-linkmessage-square193linkfedilinkarrow-up11arrow-down10
arrow-up11arrow-down1external-linkGenerative AI Is an Engineering Disaster. A shockingly inefficient trillion-dollar project.www.theatlantic.comSome_Emo_Chick@lemmy.world to Technology@lemmy.worldEnglish · 28 days agomessage-square193linkfedilink
minus-squareBrett@programming.devlinkfedilinkEnglisharrow-up0·1 day agoWhat are you using to run the model? Llama.cpp will automatically split the model between your system ram and graphics card’s vram. Qwen 3.6 is a mixture of experts model with only 3B parameters active at a time. Even without quantization your card could easily run that.
What are you using to run the model? Llama.cpp will automatically split the model between your system ram and graphics card’s vram.
Qwen 3.6 is a mixture of experts model with only 3B parameters active at a time. Even without quantization your card could easily run that.