Step3 Is padding side right or not?

I was running the script from step3: python3 train.py --step 3 --deployment-type single_gpu
The training.log shows this:

A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set padding_side='left' when initializing the tokenizer.
[2023-07-13 06:32:07,236] [INFO] [fused_optimizer.py:362:_update_scale]
Grad overflow on iteration 3
[2023-07-13 06:32:07,236] [INFO] [fused_optimizer.py:363:_update_scale] Reducing dynamic loss scale from 8192.0 to 4096.0
[2023-07-13 06:32:07,236] [INFO] [logging.py:96:log_dist] [Rank 0] Overflow detected. Skipping step. Attempted loss scale: 8192.0, reducing to 4096.0
epoch: 0|step: 7|ppo_ep: 1|act_loss: -0.9267578125|cri_loss: 2.818359375|unsuper_loss: 0.0
average reward score: -16.859375

A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set padding_side='left' when initializing the tokenizer.
epoch: 0|step: 8|ppo_ep: 1|act_loss: -0.9267578125|cri_loss: 2.818359375|unsuper_loss: 0.0
average reward score: -16.859375

A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set padding_side='left' when initializing the tokenizer.
epoch: 0|step: 9|ppo_ep: 1|act_loss: -0.9267578125|cri_loss: 2.818359375|unsuper_loss: 0.0
average reward score: -16.859375

A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set padding_side='left' when initializing the tokenizer.
epoch: 0|step: 10|ppo_ep: 1|act_loss: -0.9267578125|cri_loss: 2.818359375|unsuper_loss: 0.0
average reward score: -16.859375

A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set padding_side='left' when initializing the tokenizer.
epoch: 0|step: 11|ppo_ep: 1|act_loss: -0.9267578125|cri_loss: 2.818359375|unsuper_loss: 0.0
average reward score: -16.859375

A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set padding_side='left' when initializing the tokenizer.
epoch: 0|step: 12|ppo_ep: 1|act_loss: -0.9267578125|cri_loss: 2.818359375|unsuper_loss: 0.0
average reward score: -16.859375

A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set padding_side='left' when initializing the tokenizer.
epoch: 0|step: 13|ppo_ep: 1|act_loss: -0.9267578125|cri_loss: 2.818359375|unsuper_loss: 0.0
average reward score: -16.859375

A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set padding_side='left' when initializing the tokenizer.
epoch: 0|step: 14|ppo_ep: 1|act_loss: -0.9267578125|cri_loss: 2.818359375|unsuper_loss: 0.0
average reward score: -16.859375

A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set padding_side='left' when initializing the tokenizer.
epoch: 0|step: 15|ppo_ep: 1|act_loss: -0.9267578125|cri_loss: 2.818359375|unsuper_loss: 0.0
average reward score: -16.859375

A decoder-only architecture is being used, but right-padding was detected! For correct generation results, please set padding_side='left' when initializing the tokenizer.
epoch: 0|step: 16|ppo_ep: 1|act_loss: -0.9267578125|cri_loss: 2.818359375|unsuper_loss: 0.0
average reward score: -16.859375

And I checked the main.py, in it writes:

create common tokenizer based on actor model

tokenizer = load_hf_tokenizer(args.actor_model_name_or_path,
                              fast_tokenizer=True)
tokenizer.pad_token = tokenizer.eos_token
# make sure tokenizer is right pad in our logic
tokenizer.padding_side = 'right'

I use the dataset Dahoas/rm-static in my case. So, I wonder why is this happening? What should I do under this situation?

Start with main.py and train.py, then reproduce step 3 with Dahoas/rm-static while capturing training.log. Trace where the tokenizer is created and where generation is called, and compare the logged padding warning with the intended training behavior. Done means the expected padding configuration is documented or corrected and the warning no longer appears during a verified run.