提问:能支持单机训练百亿参数的原理是蒸馏吗?那如果用自定义算法是不是也可以单机训练?又如何自定义算法?
Open
- Dominant language
- Python
- Stars
- 2.2k
- Forks
- 257
- PR merge metrics
- No merged PRs in 30d
Description
chatGpt训练算法从transformer变成ppo以后。可以看到ai的效果和gpt3有了本质的提升。而强化学习中ppo算法还有很多提升空间。也还有不同的强化算法也有提升空间td3,deepnash。那么如果使用这些算法,并且加上集群的功能。那还了得。easyNLP可以自定义算法吗?怎么定义?又支持集群训练吗?怎么处理?以及能支持单机训练百亿参数的原理是蒸馏吗?
Contributor guide
No contributing guide indexed for this repository
Research direction
The issue names no files or tests. Start by reviewing EasyNLP's Python/PyTorch training entry points and documentation to determine whether custom PPO, TD3-like, or other algorithms and distributed training are supported. Done would require a concrete, documented answer about customization, cluster training, and the basis for single-machine training of very large models.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- distributed-systems, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100