Create auto-tuning script to find best parameter combinations
- Dominant language
- Python
- Stars
- 18
- Forks
- 8
- PR merge metrics
- No merged PRs in 30d
Description
**Is your feature request related to a problem? Please describe.**
Say we fix the hardware setup, cache length, and chunk size. What is the best way to use the available GPU memory?
We can find this out by something like HPO, or random search.
**Describe the solution you'd like**
- Define evaluation metric. Should catch OOM and output that. Otherwise, train on longest batch (or on 2?), output running time, maybe estimate of max GPU usage.
- Then, need to wrap this into some basic HPO system, like Optuna. Simpler: Just do random search
- Also, need to define a configuration space and sampler.
This would a great task for LLM prompt-driven development.
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by locating the training configuration and evaluation flow, then clarify the metric, configuration space, sampler, and handling of OOM, runtime, and GPU-memory output. Done means an agreed tuning approach can evaluate configurations and report the requested results.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100