Chat
Answer questions and follow instructions in a conversation, on one 24 GB card.
Also called “llm”, “language model”, “chatbot”, “local llm”, “gguf”, “ollama”, “llama.cpp”, “ai assistant”.
Variants
Every chat model
| Model | Accuracy (paper under it) | 4090 decode | VRAM peak | License |
|---|---|---|---|---|
| Gemma 42 sizes: 12b-q4_0, 31b-q4_0 | 89.3 IFEval | 97.4 tok/s | 7.7 GiB | Commercial OK |
| Qwen32 sizes: 0.6b-q8_0, 8b-q4_k_m | 57.3 IFEval54.5 IFEval strict prompt paper | 533 tok/s | 1.9 GiB | Commercial OK |
| Qwen3.51 size: 9b-q4_k_m | 83.5 IFEval91.5 IFEval (thinking mode not stated) paper | 134 tok/s | 5.6 GiB | Commercial OK |
| Qwen3.81 size: 27b-q4_k_m | 81.0 IFEval | 43.0 tok/s | 18.3 GiB | Commercial OK |