Elon Musk's AI assistant Grok boasted that the billionaire had the "potential to drink piss better than any human in history," among other absurd claims.
I once tried a community version from huggingface (distilled), which worked quite well even on modest hardware. But that was a while ago. Unfortunately, I haven’t had much time to look into this stuff lately, but I wanted to check that again at some point.
You can run GLM Air on pretty much any gaming desktop with 48GB+ of RAM. Check out ubergarm’s ik_llama.cpp quants on Huggingface; that’s state of the art right now.
naw, I mean more that the kind of people who uncritically would take everything a chatbot says a face value are probably better off being in chatGPTs little curated garden anyway. Cause people like that are going to immediately get grifted into whatever comes along first no matter what, and a lot of those are a lot more dangerous to the rest of us that a bot that won’t talk great replacement with you.
Ahh, thank you—I had misunderstood that, since Deepseek is (more or less) an open-source LLM from China that can also be used and fine-tuned on your own device using your own hardware.
There you go. Any of these things is just another datapoint. You need many datapoints to decide if the information you’re getting is valuable and valid.
It’s why I trust my random unauditable chinese matrix soup over my random unauditable american matrix soup frankly
You mean Deepseek on a local device?
Most aren’t really running Deepseek locally. What ollama advertises (and basically lies about) is the now-obselete Qwen 2.5 distillations.
…I mean, some are, but it’s exclusively lunatics with EPYC homelab servers, heh. And they are not using ollama.
Thx for clarifying.
I once tried a community version from huggingface (distilled), which worked quite well even on modest hardware. But that was a while ago. Unfortunately, I haven’t had much time to look into this stuff lately, but I wanted to check that again at some point.
Also, I’m a quant cooker myself. Say the word, and I can upload an IK quant more specifically tailored for whatever your hardware/aim is.
Thank you! I might get back to you on that sometime.
Do it!
Feel free to spam me if I don’t answer at first. I’m not ignoring you; Lemmy fails to send me reply notifications, sometimes.
You can run GLM Air on pretty much any gaming desktop with 48GB+ of RAM. Check out ubergarm’s ik_llama.cpp quants on Huggingface; that’s state of the art right now.
naw, I mean more that the kind of people who uncritically would take everything a chatbot says a face value are probably better off being in chatGPTs little curated garden anyway. Cause people like that are going to immediately get grifted into whatever comes along first no matter what, and a lot of those are a lot more dangerous to the rest of us that a bot that won’t talk great replacement with you.
Ahh, thank you—I had misunderstood that, since Deepseek is (more or less) an open-source LLM from China that can also be used and fine-tuned on your own device using your own hardware.
Trusting any of that shit is the problem.
There you go. Any of these things is just another datapoint. You need many datapoints to decide if the information you’re getting is valuable and valid.