labels = batch_encoded_inputs.pop('labels') KeyError: 'labels'

之前是可以训练的，使用了内置多轮规划器ThinkingModelTipsScheduler的run方法对轨迹数据进行拆分后就报错了。怀疑是pop有问题

[rank28]: Traceback (most recent call last):
[rank28]:   File "/home/ma-user/modelarts/user-job-dir/ms-swift-3.13.0dev/swift/cli/rlhf.py", line 7, in <module>
[rank28]:     rlhf_main()
[rank28]:   File "/home/ma-user/modelarts/user-job-dir/ms-swift-3.13.0dev/swift/llm/train/rlhf.py", line 233, in rlhf_main
[rank28]:     return SwiftRLHF(args).main()
[rank28]:   File "/home/ma-user/modelarts/user-job-dir/ms-swift-3.13.0dev/swift/llm/base.py", line 49, in main
[rank28]:     result = self.run()
[rank28]:   File "/home/ma-user/modelarts/user-job-dir/ms-swift-3.13.0dev/swift/ray/base.py", line 170, in wrapper
[rank28]:     return func(self, *args, **kwargs)
[rank28]:   File "/home/ma-user/modelarts/user-job-dir/ms-swift-3.13.0dev/swift/llm/train/sft.py", line 210, in run
[rank28]:     return self.train(trainer)
[rank28]:   File "/home/ma-user/modelarts/user-job-dir/ms-swift-3.13.0dev/swift/llm/train/sft.py", line 258, in train
[rank28]:     trainer.train(trainer.args.resume_from_checkpoint)
[rank28]:   File "/home/ma-user/modelarts/user-job-dir/ms-swift-3.13.0dev/swift/trainers/mixin.py", line 880, in train
[rank28]:     res = super().train(*args, **kwargs)
[rank28]:   File "/home/ma-user/anaconda3/envs/swift/lib/python3.10/site-packages/transformers/trainer.py", line 2325, in train
[rank28]:     return inner_training_loop(
[rank28]:   File "/home/ma-user/anaconda3/envs/swift/lib/python3.10/site-packages/transformers/trainer.py", line 2674, in _inner_training_loop
[rank28]:     tr_loss_step = self.training_step(model, inputs, num_items_in_batch)
[rank28]:   File "/home/ma-user/modelarts/user-job-dir/ms-swift-3.13.0dev/swift/trainers/rlhf_trainer/grpo_trainer.py", line 1848, in training_step
[rank28]:     return super().training_step(model, inputs, num_items_in_batch)
[rank28]:   File "/home/ma-user/anaconda3/envs/swift/lib/python3.10/site-packages/transformers/trainer.py", line 4014, in training_step
[rank28]:     inputs = self._prepare_inputs(inputs)
[rank28]:   File "/home/ma-user/modelarts/user-job-dir/ms-swift-3.13.0dev/swift/trainers/rlhf_trainer/utils.py", line 562, in wrapper
[rank28]:     return func(self, *args, **kwargs)
[rank28]:   File "/home/ma-user/modelarts/user-job-dir/ms-swift-3.13.0dev/swift/trainers/rlhf_trainer/grpo_trainer.py", line 196, in _prepare_inputs
[rank28]:     generation_batch = self._generate_and_score_completions(generation_batch)
[rank28]:   File "/home/ma-user/modelarts/user-job-dir/ms-swift-3.13.0dev/swift/trainers/rlhf_trainer/utils.py", line 562, in wrapper
[rank28]:     return func(self, *args, **kwargs)
[rank28]:   File "/home/ma-user/modelarts/user-job-dir/ms-swift-3.13.0dev/swift/trainers/rlhf_trainer/grpo_trainer.py", line 237, in _generate_and_score_completions
[rank28]:     batch_encoded_inputs = self._prepare_batch_inputs(inputs)
[rank28]:   File "/home/ma-user/modelarts/user-job-dir/ms-swift-3.13.0dev/swift/trainers/rlhf_trainer/utils.py", line 562, in wrapper
[rank28]:     return func(self, *args, **kwargs)
[rank28]:   File "/home/ma-user/modelarts/user-job-dir/ms-swift-3.13.0dev/swift/trainers/rlhf_trainer/grpo_trainer.py", line 871, in _prepare_batch_inputs
[rank28]:     labels = batch_encoded_inputs.pop('labels')
[rank28]: KeyError: 'labels'

Your hardware and system info
swift 3.13.0dev (main分支)
ascend 910b4

Start in swift/trainers/rlhf_trainer/grpo_trainer.py at _prepare_batch_inputs, line 871, and trace the inputs produced after ThinkingModelTipsScheduler.run splits trajectory data. Reproduce the GRPO training run on the reported setup and inspect where labels disappear before batch_encoded_inputs.pop('labels'). Done means the trajectory-splitting path no longer raises this KeyError and training proceeds.