I realize, I need to upgrade my little NUC to something bigger for higher inference of bigger llama models. I want something that you still can have on your living room’s tv bench, so no monster rack please, but that has also the necessary muscle when needed for llama. Budget doesn’t matter right now, want to understand what’s good and what’s out there. Thanks
I agree. I also think that there is nothing good in for-profit AI corporations. I can recommend the book “the empire of AI” However, I personally think self hosting and having full control of the use is a bit different.