Sweep job support for pipeline component in Azure Machine Learning
- Dominant language
- Python
- Stars
- 4.6k
- Forks
- 3.5k
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 60
Description
**Related command**
`az ml component create -f XXX.yaml`
`az ml job create -f XXX.yaml`
**Is your feature request related to a problem? Please describe.**
No.
**Describe the solution you'd like**
We are implementing a Retrieval-Augmented Generation (RAG) system using Azure Machine Learning, structured into multiple components such as:
- `process-data-input`
- `retrieve_docs_from_search_index`
- `build_prompt`
- `query_llm`
- `evaluate`
We want to perform hyperparameter tuning across these components. For example, we aim to optimize parameters such as:
- The number of documents retrieved from the search index
- Different prompt templates
- Various `q` values in LLM queries
However, **AzureML Sweep Jobs currently seem to only support `command components`**, making it impossible to tune hyperparameters across components. We tried to group our `command components` in to a `pipeline component`. But sweep job does not start correctly. It would be highly beneficial if Sweep Jobs could also support `pipeline component`, allowing us to bundle our components into a single `pipeline component` and efficiently perform hyperparameter tuning.
**Describe alternatives you've considered**
Our current approach involves creating a standalone `command component` that pulls source code from all other components. However, this:
- **Defeats the purpose** of separating concerns across multiple components
- **Violates the Single Responsibility Principle** in system design
- **Increases maintenance complexity**
**Additional context**
Extending Sweep Job support to `pipeline component` would greatly improve modularity, maintainability, and efficiency in hyperparameter tuning.
Contributor guide
Assessment
This issue has not been assessed yet.