duyntnet/pydevmini1-imatrix-GGUF

Quantizations of https://huggingface.co/bralynn/pydevmini1

Open source inference clients/UIs

• llama.cpp
• KoboldCPP
• text-generation-webui
• jan
• ik_llama.cpp
• croco.cpp

Closed source inference clients/UIs

• LM Studio
• More will be added...

---

From original readme

🚀 Try It Yourself (for free)

Don't just take my word for it. Test the model right now under the exact conditions shown in the video demonstration.

---

Model Details

• Model Type: Causal Language Model
• Number of Parameters: 4.0B
• Number of Parameters (Non-Embedding): 3.6B
• Number of Layers: 36
• Number of Attention Heads (GQA): 32 for Q, 8 for KV
• Context Length: 262,144 tokens (native)

Recommended Inference Parameters

For best results, I suggest using the following generation parameters:

• Temperature: 0.7
• Top P: 0.8
• Top K: 20
• Min P: 0.0