The chatbot still remains the most popular AI assistant worldwide with over 1.1 billion monthly users, followed by Gemini with 662 million and Claude with 245 million.
You don’t need 170+ GB of VRAM. Whole model can be run at around 1 token/second on a modern hardware from an ssd. Which is slow, don’t get me wrong, but it still somewhat useable.
You don’t need 170+ GB of VRAM. Whole model can be run at around 1 token/second on a modern hardware from an ssd. Which is slow, don’t get me wrong, but it still somewhat useable.
“Somewhat” is doing a lot of heavy lifting there 😂 How much time does it take to process your average request?
1 token/second isn’t remotely acceptsble lol