| | --- |
| | license: apache-2.0 |
| | language: |
| | - en |
| | base_model: |
| | - darwinkernelpanic/DiffReaper-3 |
| | --- |
| | # DiffReaper-Talk |
| |
|
| | A 1.5B parameter Discrete Diffusion Language Model (dLLM) optimized for parallel token prediction. Trained during foundational pre-training phase on general text corpora. |
| |
|
| | ## Summary |
| | DiffReaper-Talk uses a Transformer-based discrete diffusion architecture to predict multiple tokens in parallel. This approach avoids the sequential bottleneck of standard autoregressive generation. |
| |
|
| | ## Technical Details |
| | - **Architecture:** 24-Layer Transformer Encoder |
| | - **Embedding Dim:** 2048 |
| | - **Heads:** 16 |
| | - **Parameters:** ~1.5 Billion |
| | - **Hardware:** 1x NVIDIA A100 (80GB VRAM) |
| | - **Objective:** Markovian Discrete Denoising (Continuous Embedding Space) |
| | - **Precision:** Mixed BF16 |
| | - **Context Window:** 1024 Tokens |
| |
|
| | ## Current Status |
| | Phase 2 (Logic) Complete. Logic and domain-specific training (Code) to be applied post-convergence. |
