Define structure

# LLM Fine-Tuning Repository

This repository contains various scenarios and scripts for fine-tuning large language models (LLMs) using Azure OpenAI and open-source LLM frameworks. Each scenario is designed for specific applications and includes data preparation, training, and testing scripts.

## Repository Structure

repo-root/
│
├── azure_openai/
│ ├── direct_preference_optimization/
│ │ ├── data/
│ │ │ ├── data_generation_script.py
│ │ │ └── sample_data.csv
│ │ ├── training/
│ │ │ └── train_model.py
│ │ ├── testing/
│ │ │ └── test_model.py
│ │ └── README.md
│ ├── sql_fine_tuning/
│ │ ├── data/
│ │ ├── training/
│ │ ├── testing/
│ │ └── README.md
│ └── raft/
│ ├── data/
│ ├── training/
│ ├── testing/
│ └── README.md
│
└── opensource_llm/
├── scenario1/
│ ├── data/
│ ├── training/
│ ├── testing/
│ └── README.md
├── scenario2/
├── scenario3/
└── ...

### 1. Direct Preference Optimization (@anildwarepo @hezhang33)
- **Application**: Tailoring models to better understand user preferences.
- **When to Use**: When the model needs to be adjusted to prioritize outputs based on user feedback.
- **Good For**: Personalization, recommendation systems.
- **Folder Structure**: Includes data generation scripts, training scripts, and testing scripts.

### 2. SQL Fine-Tuning (James)
- **Application**: Enhancing model understanding of SQL queries for database management tasks.
- **When to Use**: When the model is used for generating or understanding SQL queries.
- **Good For**: Automated query generation, database management assistants.
- **Folder Structure**: Contains necessary scripts for data processing, model training, and testing.

### 3. Retrieval-Augmented Fine-Tuning (RAFT)
- **Application**: Integrating retrieval mechanisms with LLMs to provide additional contextual information during fine-tuning.
- **When to Use**: When the task requires external knowledge or context that the model needs to access dynamically.
- **Good For**: Tasks such as open-domain question answering, information retrieval, and context-rich dialogue systems.
- **Folder Structure**:
- **data/**: Scripts and datasets for retrieval augmentation, including configuration for retrieval systems.
- **training/**: Scripts that implement the fine-tuning process with retrieval mechanisms.
- **testing/**: Scripts to evaluate model performance with and without retrieval augmentation.
- **README.md**: Documentation detailing the setup and use of retrieval systems in conjunction with LLMs.

## Open-Source LLM Scenarios

### Scenario 1
- **Application**: [Describe the application for this scenario]
- **When to Use**: [Describe when to use this scenario]
- **Good For**: [Describe what this scenario is good for]
- **Folder Structure**: Includes data, training, and testing scripts specific to the scenario.

### Scenario 2
- [Similar structure as Scenario 1]

### Scenario 3
- [Similar structure as Scenario 1]

## How to Use This Repository

1. **Navigate**: Choose the scenario you are interested in from either `azure_openai` or `opensource_llm`.
2. **Prepare Data**: Use the data generation scripts in the `data` folder to prepare your dataset.
3. **Train Model**: Use the scripts in the `training` folder to fine-tune your model.
4. **Test Model**: Evaluate the performance using the scripts in the `testing` folder.
5. **Read Documentation**: Refer to the `README.md` in each scenario folder for detailed instructions and explanations.