Qwen is fine imo. Even a small model can be pretty helpful. I run a 27b on an old A6000 at work and have a 7b running locally on a home setup and both do fine with almost anything I throw at them. With the 7b, it can recognise that it’s struggling with a particular task and offload to a heavier cloud model (azure foundry)
This gives you good ROI for 99.9% of your inference and the ability to still complete big and complex tasks.
Qwen is fine imo. Even a small model can be pretty helpful. I run a 27b on an old A6000 at work and have a 7b running locally on a home setup and both do fine with almost anything I throw at them. With the 7b, it can recognise that it’s struggling with a particular task and offload to a heavier cloud model (azure foundry) This gives you good ROI for 99.9% of your inference and the ability to still complete big and complex tasks.