# ViMax

**Repository Path**: hackerwjz/ViMax

## Basic Information

- **Project Name**: ViMax
- **Description**: https://github.com/HKUDS/ViMax.git
- **Primary Language**: Unknown
- **License**: MIT
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-11-27
- **Last Updated**: 2025-11-27

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

<div align="center">
  <img src="./assets/vimax.png" width="20%"> 

  <h1 align="center">ViMax: Agentic Video Generation</h1>

  <div align="center">
  </div>

  <p align="center">
    <img src="https://img.shields.io/badge/🐍Python-3.12-00d9ff?style=for-the-badge&logo=python&logoColor=white&labelColor=1a1a2e">
	<a href="https://github.com/astral-sh/uv"><img src="https://img.shields.io/badge/⚡uv-Ready-ff6b6b?style=for-the-badge&logo=python&logoColor=white&labelColor=1a1a2e"></a>
	<img src="https://img.shields.io/badge/License-MIT-4ecdc4?style=for-the-badge&logo=opensourceinitiative&logoColor=white" alt="MIT License">
  </p>

  <p align="center">
    <a href="./Communication.md"><img src="https://img.shields.io/badge/💬Feishu-Group-07c160?style=for-the-badge&logoColor=white&labelColor=1a1a2e"></a>
    <a href="./Communication.md"><img src="https://img.shields.io/badge/WeChat-Group-07c160?style=for-the-badge&logo=wechat&logoColor=white&labelColor=1a1a2e"></a>
  </p>

  <p align="center">
    <a href='https://www.youtube.com/@AI-Creator-is-here'><img src='https://badges.aleen42.com/src/youtube.svg' /></a>&nbsp;
    <a href='https://space.bilibili.com/3546868449544308'><img src="https://img.shields.io/badge/bilibili-00A1D6.svg?logo=bilibili&logoColor=white" /></a>
  </p>

</div>
<div align="center">
	
<p>
	<a href="readme.md"><img src="https://img.shields.io/badge/English-1a1a2e?style=for-the-badge"></a>
    <a href="README_ZH.md"><img src="https://img.shields.io/badge/中文版-1a1a2e?style=for-the-badge"></a>
</p>
    <a href="#quick-start" style="text-decoration: none;">
      <img src="https://img.shields.io/badge/Quick%20Start-Get%20Started%20Now-FFC107?style=for-the-badge&logo=rocket&logoColor=white&labelColor=1a1a2e">
    </a>

</div>

---

<div align="center">



https://github.com/user-attachments/assets/5bad46b2-8276-4e1d-9480-3522640744b2




</div>

---


## 📑 目录

- [💡 核心特性](#核心特性)
- [🔮 演示示例](#演示示例)
- [🏗️ 系统架构](#️-系统架构)
- [🚀 快速开始](#快速开始)

---
## 💡核心特性

<br/>

<table align="center" width="100%" style="border: none; table-layout: fixed;">
<tr>
<td width="25%" align="center" style="vertical-align: top; padding: 20px;">

<div style="height: 80px; display: flex; align-items: center; justify-content: center;">
<h3 style="margin: 0; padding: 0;">🌟 <strong>创意到视频</strong></h3>
</div>

<div align="center" style="margin: 15px 0;">
  <img src="https://img.shields.io/badge/创意-生成-ff6b6b?style=for-the-badge&logo=algorithm&logoColor=white" alt="算法徽章" />
</div>

<div style="height: 80px; display: flex; align-items: center; justify-content: center;">
<p align="center"><strong>从灵感到银幕</strong></p>
</div>

<div style="height: 60px; display: flex; align-items: center; justify-content: center;">
<p align="center">通过智能多智能体工作流，将<strong>原始创意</strong>自动转化为完整视频故事，涵盖<strong>叙事构建、角色设计与视频制作</strong>全流程。
</p>
</div>



</td>
<td width="25%" align="center" style="vertical-align: top; padding: 20px;">

<div style="height: 80px; display: flex; align-items: center; justify-content: center;">
<h3 style="margin: 0; padding: 0;">🎨 <strong>小说到视频</strong></h3>
</div>

<div align="center" style="margin: 15px 0;">
  <img src="https://img.shields.io/badge/小说-改编-4ecdc4?style=for-the-badge&logo=book&logoColor=white" alt="前端徽章" />
</div>

<div style="height: 80px; display: flex; align-items: center; justify-content: center;">
<p align="center"><strong>智能文学改编引擎</strong></p>
</div>

<div style="height: 60px; display: flex; align-items: center; justify-content: center;">
<p align="center">将<strong>完整小说</strong>智能压缩并转化为<strong>分集视频内容</strong>，实现角色追踪、叙事压缩与逐场景视觉化改编。</p>
</div>



</td>
<td width="25%" align="center" style="vertical-align: top; padding: 20px;">

<div style="height: 80px; display: flex; align-items: center; justify-content: center;">
<h3 style="margin: 0; padding: 0;">⚙️ <strong>剧本到视频</strong></h3>
</div>

<div align="center" style="margin: 15px 0;">
  <img src="https://img.shields.io/badge/剧本-改编-9b59b6?style=for-the-badge&logo=server&logoColor=white" alt="后端徽章" />
</div>

<div style="height: 80px; display: flex; align-items: center; justify-content: center;">
<p align="center"><strong>无限剧本视频创作</strong></p>
</div>

<div style="height: 60px; display: flex; align-items: center; justify-content: center;">
<p align="center">自由创作<strong>任意剧本</strong>——从个人故事到史诗冒险，全面掌控视觉叙事的每个细节。</p>
</div>





</td>
<td width="25%" align="center" style="vertical-align: top; padding: 20px;">

<div style="height: 80px; display: flex; align-items: center; justify-content: center;">
<h3 style="margin: 0; padding: 0;">🤳 <strong>智能客串</strong></h3>
</div>

<div align="center" style="margin: 15px 0;">
  <img src="https://img.shields.io/badge/互动-生成-FFC107?style=for-the-badge&logo=server&logoColor=white" alt="后端徽章" />
</div>

<div style="height: 80px; display: flex; align-items: center; justify-content: center;">
<p align="center"><strong>用你的照片生成视频</strong></p>
</div>

<div style="height: 60px; display: flex; align-items: center; justify-content: center;">
<p align="center"><strong>创建属于你的客串视频</strong>，将自己融入无限创意剧本、电影级镜头与互动剧情中，成为故事中的明星角色。</p>
</div>



</td>
</tr>
</table>

<br/>


---
<table>
<tr>

<td align="center" width="33%">
  <video src="https://github.com/user-attachments/assets/c2fb27b0-218c-4976-b3d6-2abf8ea06be7" controls width="100%"></video>
</td>
<td align="center" width="33%">
  <video src="https://github.com/user-attachments/assets/bfa566a8-688d-4d53-a9e2-6cedeb4a399d" controls width="100%"></video>
</td>
<td align="center" width="33%">
  <video src="https://github.com/user-attachments/assets/49f61134-4f78-4285-9a9e-bb5e3e0c4abf" controls width="100%"></video>
</td>
</tr>
<tr>
<td align="center" width="33%">
  <video src="https://github.com/user-attachments/assets/a950f449-a15c-449b-a1b8-c393951aa9be" controls width="100%"></video>
</td>
<td align="center" width="33%">
  <video src="https://github.com/user-attachments/assets/bb3ff0fd-9433-4806-886a-3f77b61d06ec" controls width="100%"></video>
</td>
<td align="center" width="33%">
  <video src="https://github.com/user-attachments/assets/5bc0f395-4b3f-4b12-9387-1793d49cf78c" controls width="100%"></video>
</td>
</tr>

<tr>
<td align="center" width="33%">
  <video src="https://github.com/user-attachments/assets/5dbb80f7-aff0-4211-940c-a898f91fb80c" controls width="100%"></video>
</td>
<td align="center" width="33%">
  <video src="https://github.com/user-attachments/assets/cc0b0bcd-e7db-4839-950b-0b03949637bd" controls width="100%"></video>
</td>
<td align="center" width="33%">
  <video src="https://github.com/user-attachments/assets/85919b59-80f0-461a-af7e-a93d3fb412fc" controls width="100%"></video>
</td>
</tr>



















</table>



---

### 🎯 **端到端视频创作引擎**

**面临的挑战**：

- 🌅 **参考图像**：获取、整理并精准对齐能准确表达角色、物体、位置与环境的参考帧，耗时费力。

- 🫠 **一致性校验**：即使提供了正确的角色、位置、环境参考图与提示词，图像生成器有时仍会产出不可用图像。

- 📄 **剧本生成**：专业高质量视频需要高信息密度与结构化设计。

- 📝 **分镜设计**：将故事转化为视觉叙事，需要摄影、构图与视觉叙事的专业知识，而大多数创作者并不具备。

- 🎬 **镜头设计**：在复杂场景中保持叙事连贯性的同时，设计合理的镜头角度、转场与节奏。

- 🎨 **风格一致性**：在长视频中确保数百个镜头的角色外观、环境与艺术风格保持一致。

- ⏱️ **制作效率**：传统视频制作依赖多个专业人员与冗长流程，阻碍了独立创作者与快速原型开发。

- 🎥 **AI视频扩展性**：AI生成视频通常仅几秒，而分钟级甚至小时级的高质量长视频需要复杂的跨场景连续性与多分镜协同处理能力。

**ViMAX**：通过自动化从叙事输入到最终视频输出的完整流程，彻底消除上述制作瓶颈。

---


### 🔥 **为什么选择 ViMax？**

| 🧠 **一键生成** | 🚀 **完全创作自由** | 🔊 **音画同步** | 🎨 **专业品质** | 🤩 **互动视频** 
|:---:|:---:|:---:|:---:|:---:|
| 一句话生成完整视频 | 任何叙事皆可成真 | 音画完美融合 | 电影级输出 | 生成你的专属客串视频
| 无需技术细节——只需描述你的创意，ViMax 自动完成剧本生成、分镜设计、镜头规划、参考管理与一致性验证 | 创意无边界——无论是预告片、短篇故事、小说章节还是原创概念，ViMax 都能智能构建叙事并设计镜头语言，将任何想法变为现实 | 无缝融合角色语音与音效，打造沉浸式视听体验 | 自动质量控制确保角色一致性、场景构图合理、每帧画面均达专业水准 | 上传你的照片即可在自己的故事中互动出演——ViMax 智能将你作为角色融入视频，保持外观一致并实现自然交互



---


## 🏗️ 系统架构

### 📊 **系统概览**

**ViMax** 是一个多智能体视频生成框架，支持自动化多镜头视频生成，并确保角色与场景的一致性。系统能将你的创意无缝转化为对应视频，让你专注于讲故事，而非技术实现。

🎯 **技术能力**：

🧬 **智能长剧本生成**  
基于 RAG 的长剧本引擎，可智能分析小说级长文本，并自动切分为多场景剧本格式，精准保留关键情节与角色对话。

🪄 **表现力分镜设计**  
基于用户需求与目标受众，运用电影语言生成富有表现力的镜头级分镜，为后续视频生成奠定叙事节奏。

🔮 **多机位拍摄模拟**  
模拟多机位拍摄，提供沉浸式观看体验，同时确保同一场景内角色位置与背景的一致性。

🧸 **智能参考图选择**  
智能选取当前视频首帧所需的参考图（包括前序时间线中的分镜），确保视频越长，多角色与环境元素越准确。

⚙️ **自动化图像生成**  
基于所选参考图与前序时间线的视觉逻辑，自动生成图像生成器提示词，合理安排角色与环境的空间交互位置。

✅ **图像生成一致性校验**  
并行生成多张图像，并通过 MLLM/VLM 选择最一致的图像作为首帧，模拟人类创作者的工作流程。

⚡ **高效并行镜头生成**  
对同一机位拍摄的连续镜头进行并行处理，极大提升视频生产效率。





### 🤖 <strong>多智能体视频生成流水线</strong>

<div align="center">
  <table align="center" width="100%" style="border: none; border-collapse: collapse;">
    <tr>
      <td colspan="3" align="center" style="padding: 20px; background: linear-gradient(135deg, #667eea 0%, #764ba2 100%); border-radius: 15px; color: white; font-weight: bold;">
        🧠 <strong>输入层</strong><br/>
        📝 创意/剧本/小说 • 💭 自然语言提示 • 🖼️ 参考图像 • 🎨 风格指令 • 🧩 配置参数
      </td>
    </tr>
    <tr><td colspan="3" height="20"></td></tr>
    <tr>
      <td colspan="3" align="center" style="padding: 15px; background: linear-gradient(135deg, #ff6b6b 0%, #ee5a24 100%); border-radius: 12px; color: white; font-weight: bold;">
        🧭 <strong>中央调度</strong><br/>
        智能体调度 • 阶段切换 • 资源管理 • 重试/降级逻辑
      </td>
    </tr>
    <tr><td colspan="3" height="15"></td></tr>
    <tr>
      <td align="center" style="padding: 12px; background: linear-gradient(135deg, #3742fa 0%, #2f3542 100%); border-radius: 10px; color: white; width: 50%;">
        🧾 <strong>剧本理解</strong><br/>
        <small>角色/环境提取 • 场景边界识别 • 风格意图解析</small>
      </td>
      <td width="10"></td>
      <td align="center" style="padding: 12px; background: linear-gradient(135deg, #8c7ae6 0%, #9c88ff 100%); border-radius: 10px; color: white; width: 50%;">
        🎥 <strong>场景与镜头规划</strong><br/>
        <small>分镜步骤 • 镜头列表 • 关键帧与节奏点</small>
      </td>
    </tr>
    <tr><td colspan="3" height="15"></td></tr>
    <tr>
      <td colspan="3" align="center" style="padding: 15px; background: linear-gradient(135deg, #00d2d3 0%, #54a0ff 100%); border-radius: 12px; color: white; font-weight: bold;">
        🧪 <strong>视觉资产规划</strong><br/>
        参考图选择 • 外观/风格引导 • 提示词条件化
      </td>
    </tr>
    <tr><td colspan="3" height="15"></td></tr>
    <tr>
      <!-- 左：资产索引 -->
      <td align="center" style="padding: 12px; background: linear-gradient(135deg, #e056fd 0%, #f368e0 100%); border-radius: 10px; color: white; width: 50%;">
        🗂️ <strong>资产索引</strong><br/>
        <small>帧/参考图目录 • 嵌入向量 • 复用检索</small>
      </td>
      <td width="10"></td>
      <!-- 右：一致性与连续性 -->
      <td align="center" style="padding: 12px; background: linear-gradient(135deg, #ffa726 0%, #ff7043 100%); border-radius: 10px; color: white; width: 50%;">
        ♻️ <strong>一致性与连续性</strong><br/>
        <small>角色/环境追踪 • 参考匹配 • 时序连贯性</small>
      </td>
    </tr>
    <tr><td colspan="3" height="15"></td></tr>
    <tr>
      <td colspan="3" align="center" style="padding: 15px; background: linear-gradient(135deg, #26de81 0%, #20bf6b 100%); border-radius: 12px; color: white; font-weight: bold;">
        ✂️ <strong>视觉合成与组装</strong><br/>
        图像生成 • 最佳帧选择 • 首尾帧→视频 • 剪辑与时间线合成
      </td>
    </tr>
    <tr><td colspan="3" height="15"></td></tr>
    <tr>
      <td colspan="3" align="center" style="padding: 20px; background: linear-gradient(135deg, #045de9 0%, #09c6f9 100%); border-radius: 15px; color: white; font-weight: bold;">
        🚀 <strong>输出层</strong><br/>
        🖼️ 帧图像 • 🎞️ 片段与最终视频 • 📜 日志 • 📦 工作目录产物
      </td>
    </tr>
  </table>
</div>



## 🚀Quick Start

### 🖥️ **Environment**

```
OS: Linux, Windows
```

### 📥 **Clone and Install**
We use uv to manage the environment. For uv installation, please refer to the https://docs.astral.sh/uv/getting-started/installation/.
```bash
git clone https://github.com/HKUDS/ViMax.git
cd ViMax
uv sync
```


### 🎯 **Usage**
main_idea2video.py is used to convert your ideas into videos.
You need to configure the model and API key information in the configs/idea2video.yaml file, including three parts—the chat model, the image generator, and the video generator, as shown below
```yaml
chat_model:
  init_args:
    model: google/gemini-2.5-flash-lite-preview-09-2025
    model_provider: openai
    api_key: <YOUR_API_KEY>
    base_url: https://openrouter.ai/api/v1

image_generator:
  class_path: tools.ImageGeneratorNanobananaGoogleAPI
  init_args:
    api_key: <YOUR_API_KEY>

video_generator:
  class_path: tools.VideoGeneratorVeoGoogleAPI
  init_args:
    api_key: <YOUR_API_KEY>

working_dir: .working_dir/idea2video
```

Then, provide a simple yet thoughtful idea and the corresponding creative requirements in main_idea2video.py.
```bash
idea = \
"""
If a cat and a dog are best friends, what would happen when they meet a new cat?
"""
user_requirement = \
"""
For children, do not exceed 3 scenes.
"""
style = "Cartoon"
```

main_script2video.py generates a video based on a specific script.
You similarly need to set up the API configuration in configs/script2video.yaml file. Then, provide a scene script and the corresponding creative requirements in main_script2video.py, as shown below.
```python
script = \
"""
EXT. SCHOOL GYM - DAY
A group of students are practicing basketball in the gym. The gym is large and open, with a basketball hoop at one end and a large crowd of spectators at the other end. John (18, male, tall, athletic) is the star player, and he is practicing his dribble and shot. Jane (17, female, short, athletic) is the assistant coach, and she is helping John with his practice. The other students are watching the practice and cheering for John.
John: (dribbling the ball) I'm going to score a basket!
Jane: (smiling) Good job, John!
John: (shooting the ball) Yes!
...
"""
user_requirement = \
"""
Fast-paced with no more than 20 shots.
"""
style = "Animate Style"
```


---