MegEngine / MegEngine/Models

请问有没有 GPT2-ML 预训练模型,希望结合 DTR 进行微调

Open
#100 4 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
306
Forks
96
PR merge metrics
No merged PRs in 30d

Description

## 请求
实现MegEngine实现GPT2-ML(https://github.com/imcaspar/gpt2-ml)

## 任务描述
实验项目微调GPT2-ML,由于GPU资源不足(RTX 3060-12GB)无法调试,通过【机器之心】得知MegEngine1.4 DTR技术解决了动态图显存优化的问题,想请问有没有参考教程让我们能通过DTR实现对GPT2-ML的微调

## 目标
采用MegEngine实现的GPT2-ML模型,通过DTR进行微调

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the GPT2-ML project linked in the issue and the MegEngine DTR documentation or examples. Done means providing a MegEngine implementation of GPT2-ML together with a reference tutorial for fine-tuning it with DTR on the stated hardware constraints.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.