michaelfeil / michaelfeil/infinity
Abstraction for `resolve_torch_dtype_device(dtype: Dtype, device: Device) -> tuple[quantization_type, torch.device, torch.dtype]`
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 206
- PR merge metrics
- No merged PRs in 30d
Description
### Feature request
Too much boilerplate template:
Resolves loading, quantization, and device
Eg. if
device: auto -> torch.cuda.is_available() -> cuda or mps.
dtype: float32 -> float32, no quantization
dtype: float16 -> float16, no quantization
dtype: bfloat16 -> float16, no quantization
dtype: auto -> (bfloat16 if possible else float16) if device is cuda else float32, no quantization
dtype: int8 -> float32, int8 quantization
dtype: fp8 -> float32, fp8 quantization
### Motivation
-
### Your contribution
-
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named. Start by locating the existing loading, quantization, and device-resolution paths, then verify that the requested dtype and device cases are defined and covered by tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100