# bisheng-rt

**Repository Path**: felix-zhang/bisheng-rt

## Basic Information

- **Project Name**: bisheng-rt
- **Description**: bisheng-rt
- **Primary Language**: C++
- **License**: Apache-2.0
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2024-05-23
- **Last Updated**: 2024-05-23

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

## What is bisheng-rt?

bisheng-rt is an open source inference serving framework that power the model inference and resource allocation.
bisheng-rt makes the different model efficient deployment and provides a consistent user experience regardless of any model types.

The project is a sub-project of [bisheng](https://github.com/dataelement/bisheng).

## Key features

- High performance
- Comaptible with most computing cards (nv, atlas, cambricon, enflame) 
- Friendly model management
- Easily integrate for new model

## Quick start

### Start With Bisheng Platform

Use in Bisheng Platform [Model Manager](https://m7a7tqsztt.feishu.cn/wiki/KAl2wfI9nippfDktNKdclIIMnKf)

### Start with DataElem Services.

We provide a open cloud service for easily use. 
See [free trial](https://m7a7tqsztt.feishu.cn/wiki/HpIewJMBIiIPmMkg6T1czihrnWZ).

### Install bisheng-rt

- [Bisheng Install Manual](https://dataelem.feishu.cn/wiki/BSCcwKd4Yiot3IkOEC8cxGW7nPc)
- [Compiling bisheng-rt](https://dataelem.feishu.cn/wiki/Md6qwpEQTiHmuek22EXcm3MrnFh)

### Using from pre-builded image

[Images Release Notes](https://dataelem.feishu.cn/wiki/HpIewJMBIiIPmMkg6T1czihrnWZ)

## Documentation

For guidance on installation, development, deployment, and administration, 
check out [bisheng-rt Dev Docs](https://dataelem.feishu.cn/wiki/HpIewJMBIiIPmMkg6T1czihrnWZ). 

## Issues

Reporting problems, asking questions
We appreciate any feedback, questions or bug reporting regarding this project. 

User can post [Issues](https://github.com/dataelement/bisheng/issues), 
follow the process outlined in the [Stack Overflow document](https://stackoverflow.com/help/mcve). 

For questions, we recommend posting in our community GitHub [Discussions](https://github.com/dataelement/bisheng/discussions).


## Acknowledgments

bisheng-rt adopts dependencies from the following:

- Thanks to [triton-inference-server](https://github.com/triton-inference-server/server) for the basic framework.