philsupertramp / philsupertramp/refinebert
Train with EOS tokens
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 0
- Forks
- 0
- PR merge metrics
- No merged PRs in 30d
Description
Let's do two things
- Train a base model more aggressively (higher LR)
- Append
[EOS]to all samples while training
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inspecting the repository's training configuration and sample-preparation entry points, since the issue names no files or tests. Clarify the target learning rate and how [EOS] should be represented, then verify that training uses both changes consistently and that the resulting samples and training run meet the agreed behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100