babysor / babysor/MockingBird

在训练模型时attention图无法生成明显斜线是否正常

Open
#682 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
36.9k
Forks
5.2k
PR merge metrics
No merged PRs in 30d

Description

目前只跑了7k,素材都是自己精心挑选和处理的,但是attention图始终无法呈现明显斜线,是素材的问题还是训练数没够?有没有必要等等看
![image](https://user-images.githubusercontent.com/80482907/181198128-5652af12-31fb-489d-9434-cd08e97a3b09.png)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the training configuration and the attention plot produced after 7k training steps, using the attached image as the observed result. Compare the expected diagonal pattern with the current output and determine whether the cause is the selected materials or insufficient training; done means a clear explanation supported by a reproducible comparison.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.