README.md

---
language: 
  - en
license: apache-2.0
tags:
  - text-generation
  - instruction-tuning
  - multi-task
  - reasoning
  - email
  - summarization
  - chat
  - peft
  - lora
  - qwen
  - deepseek
base_model: deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
datasets:
  - HuggingFaceTB/smoltalk
  - snoop2head/enron_aeslc_emails
  - lucadiliello/STORIES
  - abisee/cnn_dailymail
  - wiki40b
model_type: causal-lm
inference: true
library_name: peft
pipeline_tag: text-generation
---

# 🧠 Deepseek-R1-multitask-lora

**Author:** Gilbert Akham  
**License:** Apache-2.0  
**Base model:** [`deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B`](https://huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B)  
**Adapter type:** LoRA (PEFT)  
**Capabilities:** Multi-task generalization & reasoning  

---

# 🚀 What It Can Do

This multitask fine-tuned model handles a broad set of natural language and reasoning-based tasks, such as:

✉️ Email & message writing — generate clear, friendly, or professional communications.

📖 Story & creative writing — craft imaginative narratives, poems, and dialogues.

💬 Conversational chat — maintain coherent, context-aware conversations.

💡 Explanations & tutoring — explain technical or abstract topics simply.

🧩 Reasoning & logic tasks — provide step-by-step answers for analytical questions.

💻 Code generation & explanation — write and explain Python or general programming code.

🌍 Translation & summarization — translate between multiple languages or condense information.

The model’s multi-domain training (based on datasets like SmolTalk, Everyday Conversations, and reasoning-rich samples) makes it suitable for assistants, chatbots, content generators, or educational tools.
---

## 🧩 Training Details

| Parameter | Value |
|------------|-------|
| Base model | `deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B` |
| Adapter | LoRA (r=8, alpha=32, dropout=0.1) |
| Max sequence length | 1024 |
| Learning rate | 3e-5 (cosine decay) |
| Optimizer | `adamw_8bit` |
| Grad Accumulation | 4 |
| Precision | 4-bit quantized, FP16 compute |
| Steps | 12k total (best @ ~8.2k) |
| Training time | ~2.5h on A4000 |
| Frameworks | 🤗 Transformers, PEFT, TRL, BitsAndBytes |

---

## 🧠 Reasoning Capability

Thanks to integration of **SmolTalk** and diverse multi-task prompts, the model learns:
- **Chain-of-thought style reasoning**
- **Conversational grounding**
- **Multi-step logical inferences**
- **Instruction following** across domains

Example:
```text
### Task: Explain reasoning

### Input:
If a train leaves City A at 3 PM and arrives at City B at 6 PM, covering 180 km, what is its average speed?

### Output:
The train travels 180 km in 3 hours. 
Average speed = 180 ÷ 3 = 60 km/h.