# Baichuan-M2-32B-GPTQ-Int4

**Repository Path**: hf-models/Baichuan-M2-32B-GPTQ-Int4

## Basic Information

- **Project Name**: Baichuan-M2-32B-GPTQ-Int4
- **Description**: Mirror of https://huggingface.co/baichuan-inc/Baichuan-M2-32B-GPTQ-Int4
- **Primary Language**: Unknown
- **License**: Apache-2.0
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-08-11
- **Last Updated**: 2025-08-11

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

---
license: apache-2.0
tags:
- chat
library_name: transformers
language:
- en
- zh
base_model:
- Qwen/Qwen2.5-32B
---
# Baichuan-M2-32B-GPTQ-Int4

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![Hugging Face](https://img.shields.io/badge/🤗%20Hugging%20Face-Model-yellow)](https://huggingface.co/baichuan-inc/Baichuan-M2-32B)
[![M2 GPTQ-4bit](https://img.shields.io/badge/🤗%20M2%20GPTQ--4bit-Model-orange)](https://huggingface.co/baichuan-inc/Baichuan-M2-32B-GPTQ-Int4)
[![Huawei Ascend 8bit](https://img.shields.io/badge/✨%20Huawei%20Ascend%208bit-Model-green)](https://modelers.cn/models/Baichuan/Baichuan-M2-32B-W8A8)

## 🌟 Model Overview

Baichuan-M2-32B is Baichuan AI's medical-enhanced reasoning model, the second medical model released by Baichuan. Designed for real-world medical reasoning tasks, this model builds upon Qwen2.5-32B with an innovative Large Verifier System. Through domain-specific fine-tuning on real-world medical questions, it achieves breakthrough medical performance while maintaining strong general capabilities.

**Model Features:**

Baichuan-M2 incorporates three core technical innovations: First, through the **Large Verifier System**, it combines medical scenario characteristics to design a comprehensive medical verification framework, including patient simulators and multi-dimensional verification mechanisms; second, through **medical domain adaptation enhancement** via Mid-Training, it achieves lightweight and efficient medical domain adaptation while preserving general capabilities; finally, it employs a **multi-stage reinforcement learning** strategy, decomposing complex RL tasks into hierarchical training stages to progressively enhance the model's medical knowledge, reasoning, and patient interaction capabilities.

**Core Highlights:**
- 🏆 **World's Leading Open-Source Medical Model**: Outperforms all open-source models and many proprietary models on HealthBench, achieving medical capabilities closest to GPT-5
- 🧠 **Doctor-Thinking Alignment**: Trained on real clinical cases and patient simulators, with clinical diagnostic thinking and robust patient interaction capabilities
- ⚡ **Efficient Deployment**: Supports 4-bit quantization for single-RTX4090 deployment, with 58.5% higher token throughput in MTP version for single-user scenarios

## 📊 Performance Metrics

### HealthBench Scores

| Model Name | HealthBench | HealthBench-Hard | HealthBench-Consensus |
|------------|-------------|------------------|-----------------------|
| Baichuan-M2 | 60.1 | 34.7 | 91.5 |
| gpt-oss-120b | 57.6 | 30 | 90 |
| Qwen3-235B-A22B-Thinking-2507 | 55.2 | 25.9 | 90.6 |
| Deepseek-R1-0528 | 53.6 | 22.6 | 91.5 |
| GLM-4.5 | 47.8 | 18.7 | 85.3 |
| Kimi-K2 | 43 | 10.7 | 90.9 |
| gpt-oss-20b | 42.5 | 10.8 | 82.6 |

### General Performance

| Benchmark | Baichuan-M2-32B | Qwen3-32B (Thinking) |
|-----------|-----------------|-----------|
| AIME24 | 83.4 | 81.4 |
| AIME25 | 72.9 | 72.9 |
| Arena-Hard-v2.0 | 45.8 | 44.5 |
| CFBench | 77.6 | 75.7 |
| WritingBench | 8.56 | 7.90 |

*Note: AIME uses max_tokens=64k, others use 32k; temperature=0.6 for all tests.*

## 🔧 Technical Features

📗 **Technical Blog**: [Blog - Baichuan-M2](https://www.baichuan-ai.com/blog/baichuan-M2)

### Large Verifier System
- **Patient Simulator**: Virtual patient system based on real clinical cases
- **Multi-Dimensional Verification**: 8 dimensions including medical accuracy, response completeness, and follow-up awareness
- **Dynamic Scoring**: Real-time generation of adaptive evaluation criteria for complex clinical scenarios
### Medical Domain Adaptation
- **Mid-Training**: Medical knowledge injection while preserving general capabilities
- **Reinforcement Learning**: Multi-stage RL strategy optimization
- **General-Specialized Balance**: Carefully balanced medical, general, and mathematical composite training data

## ⚙️ Quick Start

For deployment, you can use `sglang>=0.4.6.post1` or `vllm>=0.9.0` or to create an OpenAI-compatible API endpoint:
- SGLang:
    ```shell
    python -m sglang.launch_server --model-path baichuan-inc/Baichuan-M2-32B-GPTQ-Int4 --reasoning-parser qwen3
    ```
To turn on kv cache FP8 quantization:
    ```shell
    python -m sglang.launch_server --model-path baichuan-inc/Baichuan-M2-32B-GPTQ-Int4 --reasoning-parser qwen3 --kv-cache-dtype fp8_e4m3 --attention-backend flashinfer
    ```

- vLLM:
    ```shell
    vllm serve baichuan-inc/Baichuan-M2-32B-GPTQ-Int4  --reasoning-parser qwen3
    ```
To turn on kv cache FP8 quantization:
    ```shell
    vllm serve baichuan-inc/Baichuan-M2-32B-GPTQ-Int4  --reasoning-parser qwen3 --kv_cache_dtype fp8_e4m3
    ```

## MTP inference with SGLang

1. Replace the qwen2.py file in the sglang installation directory with draft/qwen2.py.
2. Launch sglang:
```
python3 -m sglang.launch_server \
--model Baichuan-M2-32B-GPTQ-Int4 \
--speculative-algorithm EAGLE3 \
--speculative-draft-model-path Baichuan-M2-32B-GPTQ-Int4/draft \
--speculative-num-steps 6 \
--speculative-eagle-topk 10 \
--speculative-num-draft-tokens 32 \
--mem-fraction 0.9 \
--cuda-graph-max-bs 2 \
--reasoning-parser qwen3 \
--dtype bfloat16
```

## ⚠️ Usage Notices
1. **Medical Disclaimer**: For research and reference only; cannot replace professional medical diagnosis or treatment
2. **Intended Use Cases**: Medical education, health consultation, clinical decision support
3. **Safe Use**: Recommended under guidance of medical professionals

## 📄 License
Licensed under the [Apache License 2.0](LICENSE). Research and commercial use permitted.

## 🤝 Acknowledgements
- Base Model: Qwen2.5-32B
- Training Framework: verl
- Inference Engines: vLLM, SGLang
- Quantization: AutoRound, GPTQ

Thank you to the open-source community. We commit to continuous contribution and advancement of healthcare AI.

## 📞 Contact Us
- Resources: [Baichuan AI Website](https://www.baichuan-ai.com)
- Technical Support: [GitHub](https://github.com/baichuan-inc)

---
<div align="center">

**Empowering Healthcare with AI, Making Health Accessible to All**

</div>