EleutherAI / EleutherAI/cookbook

Add HuggingFace arg so that arch is automatic

Open
#1 3 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
852
Forks
47
PR merge metrics
No merged PRs in 30d

Description

Stas Bekman had the idea of supporting a HuggingFace model as input so that all model architecture settings don't need manually dug up. We'd like something like:

```
python transformer_mem.py --hf_model_name_or_path meta-llama/Llama-2-7b-hf --num-gpus 8 --zero-stage 3 --batch-size-per-gpu 2 --sequence-length 4096
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.