Do you host your own ML / AI / LLM? What do you use, and what do you use it for?

  • c64z86@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    18 days ago

    Yep! on my gaming Laptop I use Qwen 3.5 and Gemma 4 with an RTX 4080 Mobile, I’m also just able to run video generation on it to thanks to a quant of LTX 2.3. On this AMD AI laptop I am able to run a few LLMs via the NPU thanks to FastflowLM and the Lemonade server.

    I use it for messing about pretty much. I haven’t found any serious uses for AI so it’s still a toy.