Open R1: A Comprehensive Evaluation vs GPT-4 and Claude 3
Model Development Pipeline
The Open R1 pipeline follows a three-stage training strategy, closely aligned with the official DeepSeek-R1 methodology:
- Knowledge Distillation: A 35K-sample validation trajectory set is extracted from DeepSeek-R1 and used to construct the Mixture-of-Tohughts collection. A distilled model (OpenR1-Distill) is trained on this data using Qwen architectures.
- Reinforcement Learning Fine-Tuning: The GRPO algorithm is applied with a verifiable reward function to produce Open R1-Zero, strengthening step-wise logical coherence.
- Final Integration: Supervised fine-tuning (SFT) and RL outputs are merged to form the final Open R1 model.
Performence Benchmark Results
Mathematical Reasoning (AIME 2024)
| Model | AIME24 Accuracy |
|---|---|
| DeepSeek-R1-Distill-Qwen-1.5B | 30.7% |
| DeepSeek-R1-Distill-Qwen-7B | 50.8% |
| DeepSeek-R1-Distill-Qwen-32B | 69.7% |
Run evaluation locally:
make evaluate MODEL=deepseek-ai/DeepSeek-R1-Distill-Qwen-32B TASK=aime24
Programming Benchmakr (Codeforces CoT + IOI 2024)
The project introduces CodeForces-CoTs, a dataset with 10 k problems and 100 k reasoning traces. When fine-tuned on this corpus:
- A 7B Qwen variant outperforms Claude 3.7 Sonnet
- A 32B variant surpasses DeepSeek-R1 on IOI 2024
Comparative Strengths
- Transparency & Reproducibility: Full code, hyperparameter configs (e.g.,
grpo/config_demo.yaml), and training logs are available for inspection and adaptation. - Domain Specialization: Achieves parity or superiority to GPT-4/Claude in math and coding tasks, particularly at 32B scale.
- Extensibility: Benchmark harness (e.g.,
scripts/run_benchmarks.py) enables custom task injection and pipeline extension.
Identified Limitations
- General Knowledge: Lagging behind on GPQA Diamond (science/前沿 physics, biology, etc.), indicating a need for broader knowledge injection.
- Multilingual Support: Currently tuned for English; non-Latin and multilingual capabilities remain underexplored.
- Inference Latency: Larger variants (≥32B) show higher end-to-end response latency compared to optimized commercial inference endpoints.
Getting Started
Clone the repository and install dependencies:
git clone https://gitcode.com/gh_mirrors/open/open-r1
cd open-r1
pip install -r requirements.txt
Pretrained checkpoints and fine-tuning scripts for 1.5B/7B/32B variants are included. Documentation for評估, LoRA adaptation, and deployment (vLLM/TGI) is available in the repository’s docs/ subtree.
