awslabs / awslabs/keys_values

Create auto-tuning script to find best parameter combinations

Open
#79 6 comments 0 reactions 0 assignees View on GitHub
enhancement good first issue
Dominant language
Python
Stars
18
Forks
8
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**
Say we fix the hardware setup, cache length, and chunk size. What is the best way to use the available GPU memory?

We can find this out by something like HPO, or random search.

**Describe the solution you'd like**

- Define evaluation metric. Should catch OOM and output that. Otherwise, train on longest batch (or on 2?), output running time, maybe estimate of max GPU usage.
- Then, need to wrap this into some basic HPO system, like Optuna. Simpler: Just do random search
- Also, need to define a configuration space and sampler.

This would a great task for LLM prompt-driven development.

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points. Start by locating the training configuration and evaluation flow, then clarify the metric, configuration space, sampler, and handling of OOM, runtime, and GPU-memory output. Done means an agreed tuning approach can evaluate configurations and report the requested results.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.