在看文档时说训练sft模型时 需要将该 token 指定为<eom>，但是在哪里改呢？
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 12.3k
- Forks
- 1.1k
- PR merge metrics
- No merged PRs in 30d
Description
在看文档时说训练sft模型时 需要将该 token 指定为< eom >，但是在哪里改呢？
训练
num_machines=4
num_processes=$((num_machines * 8))
machine_rank=0
accelerate launch --config_file ./configs/sft.yaml --num_processes $num_processes --num_machines $num_machines --machine_rank $machine_rank --deepspeed_multinode_launcher standard finetune_moss.py
--model_name_or_path fnlp/moss-moon-003-sft-plugin
--data_dir ./sft_data
--output_dir ./ckpts/moss-moon-003-sft-plugin
--log_dir ./train_logs/moss-moon-003-sft-plugin
--n_epochs 2
--train_bsz_per_gpu 1
--eval_bsz_per_gpu 1
--learning_rate 0.000015
--eval_step 200
--save_step 2000"
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with finetune_moss.py and configs/sft.yaml, then compare them with the SFT command and the referenced documentation. Identify where the token is configured and update the documentation so the setting location and expected value are explicit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100