Lightning-AI / Lightning-AI/lit-llama
How to migrate the finetune/full.py to a multi-node and multi-gpu environment?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6.1k
- Forks
- 517
- PR merge metrics
- No merged PRs in 30d
Description
I would like to make use more than 1 of the servers in my lab. Is it convenient to migrate the fully fine-tuning code to a multi-node environment? Has anybody tried this? I found there was a doc: https://lightning.ai/docs/pytorch/stable/clouds/cluster_intermediate_1.html, but I think it is not detailed for me.
Thanks!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Read finetune/full.py and the linked Lightning PyTorch cluster documentation first. Determine whether the existing full-finetuning entry point supports multi-node and multi-GPU execution; done would be a documented, reproducible migration path or a clear statement of what is unsupported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- distributed-systems, machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100