# kvcache-ops

**Repository Path**: openeuler/kvcache-ops

## Basic Information

- **Project Name**: kvcache-ops
- **Description**: 本项目已经迁移至 AtomGit  || This project has been migrated to AtomGit || Linked: https://atomgit.com/openeuler/kvcache-ops
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 3
- **Forks**: 8
- **Created**: 2025-08-11
- **Last Updated**: 2025-12-25

## Categories & Tags

**Categories**: Uncategorized

**Tags**: sig-Long

## README

# Notice: This project has been migrated to [AtomGit](https://atomgit.com/openeuler/kvcache-ops)
# 通知: 本项目已经正式迁移至 [AtomGit](https://atomgit.com/openeuler/kvcache-ops) 平台
# KVCache Ops

KVCache Ops is a simple library containing LLM KVCache related operators for Ascend NPU.

We currently have a few operators that support KVCache offload/reload (D2H & H2D).

## Compilation

To compile the kernels, we leverage the ascendc_library function from the ascend_toolkit.

This means that we can leverage compilation macros like `__CCE_AICORE__` to switch between implementation at compile time for the device side.

However, we have embedded the host side execution part also in the kernels files, and therefore we introduce the host side macro `ASCEND_AICORE_ARCH` for host side compilation.

To use the kernels and compile in your application, you can try the following:

```
# CMakeLists.txt
# assume kvcache-ops is a submodule in your main application
# ...
add_subdirectory(third_party/kvcache-ops)
# ...
```

## Future work

Separate the arguments into a op host tiling data structure and modify the build step for the kernels.