I’m still messing around with self hosting llm, rn ive settled on using lumo from proton if I use an llm.

When I have run llm, I used koboldcpp. Works pretty well, depends on what you are doing and what models you use. Forget which models ive been using off the top of my head

Running decencored Qwen3.6-27b and a 9b Gemma for RAG and scrapes on Ollama with a mostly vibe coded discord bot. Just got it to run tools and scrape and post news on a schedule. The first model I can run locally that’s smart enough to be useful. May give Jan a try for the back end after reading that other guys rant.

Mostly use it for stupid questions I could have googled and to brag to friends.

I host my own AI, mostly for testing and because I wanted something that was mine and mine alone. I use Ollama and run models like Llama, Mistral, and Qwen. I honestly don’t use it much, but I wanted to have my own setup just in case online services go down or become less available. It’s part of my whole “own everything I use” mantra that I’ve been on lately.

If I wanted AI for some reason, it’d be self-host or nothing.

No, I have taste.

I prefer my critical faculties completely intact and un-altered, thank you very much.
I do not require or desire a 400 watt bullshit-artist yes-man or vulnerability coder cooking my GPU.

Running qwen3.6 27b through llama.cpp.

It’s about as capable as sonnet 3.5.

I use it for light scripting, but real coding is done by cloud models.

I’m also using it as the brain for my Hermes agent. It sends me digests of news, subreddits, chats that I’d like to read but don’t have time for. It does a great job researching things on the web for me, too.

Partially. I started with hosting my own llama3.2 + granite4 models using Ollama for my Home Assistant smart home and for general chat with OpenWebUI. I also run whisper for speech-to-text locally on my 1080 Ti GPU. I like the privacy and ownership of my self-hosted models, but I started to run into limitations with the small weights. So I built some tools that allow me to route traffic to larger models hosted on DeepInfra depending on my need. For example, to GLM/Kimi models for code reviews or for my custom harnesses or harder problems.

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:

3 acronyms in this thread; the most compressed thread commented on today has 3 acronyms.

[Thread #27 for this comm, first seen 25th Jun 2026, 15:40] [FAQ] [Full list] [Contact] [Source code]

I currently run Qwen3.6-27b on llama.cpp and use it via openwebui. Mostly, I use it for web research via tavily, to a lesser extent for coding and interactively learning about things that are new to me but common in training data (such as basic math or ML concepts).

No. I still have no use for it and everything I use is automated without at a far lower footprint.

Yes, llama-swap and I use it for home assistant text-gen notifications, basic coding tasks, etc

If anyone here self-hosts definitely check out llama-swap as it has some nifty features for hotswapping LLMs, image generation models and voice models.

I’m using anythingllm. It’s quite easy to setup and use. I’m impressed of the perf on comodity hardware.

I have a simple slow model running on CPU in my cluster for karakeep. I’ve tried running a variety of models on my 7900XT but even with 16GB their performance just isn’t there. My new work m5 Mac book with 48GB of ram is the first time I’ve seen usable performance for local models and it has been pretty impressive.

I hosted Qwen 3.5 9b uncensored on my site at https://masland.tech/ for a while. I didn’t really use it and no one else used it so I took it down. These days I’m spending most of my time finding uses for AI and accessibility. One of the next things I’m planning is a video to text reasoning system, primarily for the purpose of grading used electronic devices.