# CFPO

**Repository Path**: mirrors_microsoft/CFPO

## Basic Information

- **Project Name**: CFPO
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: MIT
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-08-06
- **Last Updated**: 2025-08-30

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# CFPO: Content-Format Integrated Prompt Optimization

This is the implementation of [Beyond Prompt Content: Enhancing LLM Performance via Content-Format Integrated Prompt Optimization](https://arxiv.org/abs/2502.04295).

CFPO (Content-Format Integrated Prompt Optimization) is a novel methodology that concurrently optimizes both prompt content and format for Large Language Models (LLMs) through an iterative refinement process. It addresses the limitations of existing prompt optimization research, which primarily focuses on prompt content while overlooking the critical dimension of prompt formatting.

![Pipeline of CFPO](assets/pipeline.png)
*CFPO iteratively optimizes prompt content and format through a two-stage process, combining case-diagnosis, Monte Carlo sampling, and dynamic format exploration, including format selection with scoring system and format generation.*


## Key Features

- **Integrated Optimization**: Jointly optimizes **prompt content** (via diagnosis and variations) and **format** (via UCT-based selection and LLM-guided generation).  

- **Format optimization strategy**: CFPO employs dynamic format optimization that generates new formats in an iterative manner and evaluates formats instance through a scoring system to select the best option. 

- **Performance**: CFPO consistently improves LLM performance in a measurable and effective manner.



## Usage  

### Installation
```shell  
git clone https://github.com/HenryLau7/CFPO.git  
```
### Quick Start  
```shell  
bash scripts/*.sh
```
Key arguments:
```shell
--task #TASK NAME #\
--output_marker #LOG MARKER# \
--train_size #TRAINING SIZE# \
--minibatch_size #FOR DIAGNOSIS# \
--valid_size #VAL SIZE# \
--test_size #TEST SIZE# \
--controller #SCHEDULAR# \
--opt_llm #OPT_LLM# \
--eval_llm #EVAL_LLM# \
--vllm_pth #VLLM_LOCAL_PATH# \
--init_temperature #INIT_TEMP# \
--rounds #ROUNDS# \
--beam_size #BEAM SIZE TO MAINTAIN# \
--num_return #RETURN NUMBER OF OPTIMIZED PROMPTS# \
--num_feedbacks #NUMBER OF PROMPTS GENERATED BY DIAGNOSIS# \
--errors_per_feedback #NUMBER OF INCORRECT SAMPLE FOR DIAGNOSIS# \
--correct_per_feedback #NUMBER OF CORRECT SAMPLE FOR DIAGNOSIS# \
--apply_per_feedback #NUMBER OF SEARCHED PROMPT PER FEEDBACK# \
--num_random 1 #NUMBER OF PROMPTS GENERATED BY MONTE-CARLO SAMPLING# \
--num_format 1 #NUMBER OF PROMPTS GENERATED BY FORMAT MUTATION# \
--select_method #SELECT METHOD FOR FORMAT# \
--gpu_id 0 #SET GPU DEVICE ID## \
```

## Intended Uses

- CFPO is best suited for researchers and developers seeking to improve the performance of LLMs across various tasks by automatically optimizing prompts. It is particularly effective for scenarios where prompt formatting significantly impacts LLM performance, especially with foundational models.
- CFPO is being shared with the research community to facilitate reproduction of our results and foster further research in this area.

## Out-of-scope Uses

- We do not recommend using CFPO in commercial or real-world applications without further testing and development. It is being released for research purposes.

- CFPO was not designed or evaluated for all possible downstream purposes. Developers should consider its inherent limitations as they select use cases, and evaluate and mitigate for accuracy, safety, and fairness concerns specific to each intended downstream use.


## Citation

If you find the code useful, please cite our paper as follows:

```bibtex
@misc{liu2025cfpo,
      title={Beyond Prompt Content: Enhancing LLM Performance via Content-Format Integrated Prompt Optimization}, 
      author={Yuanye Liu and Jiahang Xu and Li Lyna Zhang and Qi Chen and Xuan Feng and Yang Chen and Zhongxin Guo and Yuqing Yang and Cheng Peng},
      year={2025},
      eprint={2502.04295},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2502.04295}, 
}
```