# Gear Train Python

**Repository Path**: kai-zheng/gear-train-py

## Basic Information

- **Project Name**: Gear Train Python
- **Description**: 灵活的推理框架 Gear Train
- **Primary Language**: Python
- **License**: Artistic-2.0
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 2
- **Created**: 2024-08-14
- **Last Updated**: 2024-08-14

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

<p align="center">
    <br>
    <img src="assets/Geartrain.jpg" width="1200"/>
    <br>
</p>

## 介绍
GearTrain 提供了灵活的推理框架, 支持视频、图片推理方式

基于 GearTrain 用户可像齿轮一样自由组合各种Pipeline，实现各种推理任务

## Environment

```bash
export PYTHONPATH=$PYTHONPATH:/workspace/gear-train-py
```

根据需求选择不同框架, 目前已实现框架只有 `OR`

`onnxruntime`: 
```bash
pip install onnxruntime-gpu -i https://pypi.tuna.tsinghua.edu.cn/simple
```

## 框架
### Backend
#### Arguments

|Argument|Type|Default|Description
|-|-|-|-
|`framework`|`str`|`ort`|Target format for the exported model, such as 'onnx', 'torchscript', 'tensorflow', or others, defining compatibility with various deployment environments.
|`half`|`bool`|`False`|Enables FP16 (half-precision) quantization, reducing model size and potentially speeding up inference on supported hardware.
|`int8`|`bool`|`False`|Activates INT8 quantization, further compressing the model and speeding up inference with minimal accuracy loss, primarily for edge devices.
|`device`|`str`|`cpu`|Device for running backend

#### Formats

|Format|`framework` Agument|`device` Agument|Model|Arguments
|-|-|-|-|-
|ONNX|`ort`|`cpu`,`cuda`|model.onnx|`half`
|TensorRT|`engine`|`cuda`|model.engine|`half`,`int8`



### Pipeline

#### [Pipeline 文档](docs/Pipeline.md)

- 一个实例化的 `Pipeline` 也可以作为另一个 `Pipeline` 的 `model`, 从而实现管道的无限延长拼接

### 待改进

- 在一个 `pipeline` 中同时调用多个 `model`

- 流水线并行方案