dataset_mixer_list error when training model
- Ngôn ngữ chính
- Python
- Star
- 48
- Fork
- 5
- Chỉ số merge pull request
- Không có pull request nào được merge trong 30 ngày
Mô tả
Hello I'm trying to reproduce the model training with below command
```
CUDA_VISIBLE_DEVICES=1,2,3,4,5,6,7 accelerate launch \
--mixed_precision bf16 \
--num_machines 1 \
--num_processes 7 \
--use_deepspeed \
--deepspeed_config_file configs/ds_configs/stage3_offloading_accelerate.conf \
open_instruct/finetune.py \
--model_name_or_path ../pretrained/tulu-2-7b \
--use_flash_attn \
--tokenizer_name ../pretrained/tulu-2-7b \
--dataset_name allenai/SciRIFF-train-mix \
--use_slow_tokenizer \
--max_seq_length 4096 \
--preprocessing_num_workers 16 \
--per_device_train_batch_size 2 \
--gradient_accumulation_steps 8 \
--learning_rate 2e-5 \
--lr_scheduler_type linear \
--warmup_ratio 0.03 \
--weight_decay 0. \
--num_train_epochs 2 \
--output_dir output/sciriff \
--with_tracking \
--report_to tensorboard \
--logging_steps 1 \
--push_to_hub false
```
and I got this error
```
[rank0]: Traceback (most recent call last):
[rank0]: File "/workspace/open-instruct/open_instruct/finetune.py", line 927, in
[rank0]: main(args)
[rank0]: File "/workspace/open-instruct/open_instruct/finetune.py", line 460, in main
[rank0]: train_dataset = get_cached_dataset_tulu_sft(
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: File "/workspace/open-instruct/open_instruct/dataset_transformation.py", line 798, in get_cached_dataset_tulu_sft
[rank0]: assert len(dataset_mixer_list) % 2 == 0, f"Data mixer list length is not even: {dataset_mixer_list}"
[rank0]: ^^^^^^^^^^^^^^^^^^^^^^^
[rank0]: TypeError: object of type 'NoneType' has no len()
```
How can I fix this error.
P/s:
It seems that when creating a dataset object, you only use `dataset_mixer_list` but in the example command you use `dataset_name`.
```
if args.dataset_mixer is not None:
args.dataset_mixer_list = [item for pair in args.dataset_mixer.items() for item in pair]
with accelerator.main_process_first():
train_dataset = get_cached_dataset_tulu_sft(
args.dataset_mixer_list,
tc,
args.max_seq_length,
args.hf_entity,
)
train_dataset = train_dataset.shuffle(seed=args.seed)
train_dataset.set_format(type="pt")
```
Hướng dẫn đóng góp
Chưa lập chỉ mục được hướng dẫn đóng góp cho kho mã nguồn này
Đánh giá
Issue này chưa được đánh giá.