Update README.md
Browse files
README.md
CHANGED
|
@@ -34,7 +34,7 @@ We are thrilled to introduce Seed-Coder, a powerful, transparent, and parameter-
|
|
| 34 |
<img width="100%" src="imgs/seed-coder_intro_performance.jpg">
|
| 35 |
</p>
|
| 36 |
|
| 37 |
-
This repo contains the **Seed-Coder-8B-Base** model,
|
| 38 |
- Type: Causal language models
|
| 39 |
- Training Stage: Pretraining
|
| 40 |
- Data Source: GitHub data, code-related web data
|
|
@@ -112,19 +112,14 @@ print(output[0]["generated_text"])
|
|
| 112 |
|
| 113 |
## Evaluation
|
| 114 |
|
| 115 |
-
Seed-Coder-8B-Base has been evaluated across a variety of code
|
| 116 |
-
|
| 117 |
-
It demonstrates strong capabilities in:
|
| 118 |
-
- Fluent and contextually appropriate code completion.
|
| 119 |
-
- Reasoning about code structure and inferring missing logic.
|
| 120 |
-
- Generalizing across different programming languages, coding styles, and codebases.
|
| 121 |
|
| 122 |
| | DeepSeek-Coder-6.7B-Base | OpenCoder-8B-Base | Qwen2.5-Coder-7B | Seed-Coder-8B-Base |
|
| 123 |
|------------|:------------------------:|:-----------------:|:----------------:|:------------------:|
|
| 124 |
| HumanEval | 47.6 | 66.5 | 72.0 | 77.4 |
|
| 125 |
| MBPP | 70.2 | 79.9 | 79.4 | 82.0 |
|
| 126 |
| MultiPL-E | 44.7 | 61.0 | 58.8 | 67.6 |
|
| 127 |
-
|
|
| 128 |
|
| 129 |
For detailed benchmark performance, please refer to our [📑 Technical Report](https://github.com/ByteDance-Seed/Seed-Coder/blob/master/Seed-Coder.pdf).
|
| 130 |
|
|
|
|
| 34 |
<img width="100%" src="imgs/seed-coder_intro_performance.jpg">
|
| 35 |
</p>
|
| 36 |
|
| 37 |
+
This repo contains the **Seed-Coder-8B-Base** model, with the following features:
|
| 38 |
- Type: Causal language models
|
| 39 |
- Training Stage: Pretraining
|
| 40 |
- Data Source: GitHub data, code-related web data
|
|
|
|
| 112 |
|
| 113 |
## Evaluation
|
| 114 |
|
| 115 |
+
Seed-Coder-8B-Base has been evaluated across a variety of code generation, code completion, and code reasoning benchmarks, achieving state-of-the-art performance among open-source models of similar size.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 116 |
|
| 117 |
| | DeepSeek-Coder-6.7B-Base | OpenCoder-8B-Base | Qwen2.5-Coder-7B | Seed-Coder-8B-Base |
|
| 118 |
|------------|:------------------------:|:-----------------:|:----------------:|:------------------:|
|
| 119 |
| HumanEval | 47.6 | 66.5 | 72.0 | 77.4 |
|
| 120 |
| MBPP | 70.2 | 79.9 | 79.4 | 82.0 |
|
| 121 |
| MultiPL-E | 44.7 | 61.0 | 58.8 | 67.6 |
|
| 122 |
+
| CRUXEval-O | 41.0 | 43.9 | 56.0 | 48.4 |
|
| 123 |
|
| 124 |
For detailed benchmark performance, please refer to our [📑 Technical Report](https://github.com/ByteDance-Seed/Seed-Coder/blob/master/Seed-Coder.pdf).
|
| 125 |
|