Namespace Error when trying to train reward model

I ran python trainer_rm.py --configs defaults_rm oasst-rm-1-pythia-1.4b inside the model/model_training folder.

I got an error trying to train the model:

python trainer_rm.py --configs defaults_rm oasst-rm-1-pythia-1.4b
[2023-09-17 11:23:03,502] [INFO] [real_accelerator.py:158:get_accelerator] Setting ds_accelerator to cuda (auto detect)
trainig_conf = Namespace(rng_seed=2703368087, is_reward_model=True, pooling='last', learning_rate='8e-6', gradient_checkpointing=False, gradient_accumulation_steps=4, per_device_train_batch_size=1, per_device_eval_batch_size=5, adam_beta1=0.9, adam_beta2=0.95, adam_epsilon='1e-12', weight_decay=0.0, warmup_steps=50, eval_steps=500, save_steps=1000, save_strategy='steps', max_length=2048, num_train_epochs=2, logging_steps=10, max_grad_norm=2.0, save_total_limit=4, dtype='float32', eval_accumulation_steps=None, freeze_layer=None, cache_dir='.cache', loss_fn='RMLoss', score_l2_reg=0.001, eval_size=None, log_dir='base', quantization=False, seq2seqmodel=False, fuse_gelu=True, log_wandb=True, verbose=False, output_dir='.saved_models_rm', use_custom_sampler=True, residual_dropout=0.01, use_flash_attention=True, sort_by_length=False, per_digit_tokens=False, datasets_extra=[], metrics=['accuracy', 'kendalltau'], deepspeed_config='configs/zero_config.json', max_replies=5, datasets=[{'oasst_export': {'lang': 'en,es,de,fr', 'hf_dataset_name': 'OpenAssistant/oasst1', 'val_split': 0.1}}, {'augment_oasst': {'input_file_path': 'augmented_latin_cyrillic_oasst_2023-03-27.jsonl'}}, {'anthropic_rlhf': {'fraction': 0.1, 'max_val_set': 1000}}, {'shp': {'max_val_set': 1000}}, {'hellaswag': {'fraction': 0.5, 'max_val_set': 1000}}, {'webgpt': {'val_split': 0.05, 'max_val_set': 1000}}, {'hf_summary_pairs': {'fraction': 0.1, 'max_val_set': 250}}], model_name='andreaskoepf/pythia-1.4b-gpt4all-pretrain', wandb_entity='open-assistant', local_rank=-1, deepspeed=False, resume_from_checkpoint=False, show_dataset_stats=False, world_size=1)
RNG seed: 2703368087
You are using a model of type gpt_neox to instantiate a model of type gpt_neox_reward_model. This is not supported for all configurations of models and can yield errors.
Some weights of GPTNeoXRewardModel were not initialized from the model checkpoint at andreaskoepf/pythia-1.4b-gpt4all-pretrain and are newly initialized: ['out_proj.bias', 'out_proj.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Number of trainable parameters: 1311M
Traceback (most recent call last):
  File "/home/benmainbird/Projects/ml/Open-Assistant/model/model_training/trainer_rm.py", line 334, in <module>
    main()
  File "/home/benmainbird/Projects/ml/Open-Assistant/model/model_training/trainer_rm.py", line 191, in main
    model = get_model(training_conf, tokenizer)
  File "/home/benmainbird/miniconda3/envs/oa/lib/python3.10/site-packages/model_training/utils/utils.py", line 353, in get_model
    residual_dropout_lima=conf.residual_dropout_lima,
AttributeError: 'Namespace' object has no attribute 'residual_dropout_lima'. Did you mean: 'residual_dropout'?

It worked (I didn't get the error when I added residual_dropout_lima: true to the config.yaml. (But I don't know if it is supposed to be true or what it does in general)
The following attributes are missing from the Namespace:

• residual_dropout_lima
• use_system_tag
• system_property_dropout
• system_add_length

I only tried the oasst-rm-1-pythia-1.4b model, the other model sizes have to be tested.

Start with model/model_training/configs/config_rm.yaml and model/model_training/trainer_rm.py, then inspect the get_model call shown in model_training/utils/utils.py. Run the reported reward-model command and compare the parsed Namespace with the four missing attributes listed in the issue. Done means the command can initialize and begin training for the tested model sizes without missing configuration attributes.