[REQUEST] HF generate() enabling when training with pipeline

Is your feature request related to a problem? Please describe.
Deepspeed is a very useful library and I'm now training LlamaForCausalLM in HF with pipeline parallel. I notice that when eval during training procedure, I should use engine.eval_batch(). But I want to generate texts using HF's powerful generate() which includes a variety of methods for sampling during generation.

Describe the solution you'd like
When eval HF's CausalLM during training with pipeline_engine, I hope to use generate().

Describe alternatives you've considered
Now, I inherits the GenerationMixin class in HF init with engine, and get logits in forward() by engine.eval_batch(return_logits=True). After obtaining the engine, it is not very elegant to require initializing a new class for generation during evaluation.

Thanks in advance.

Start by tracing how pipeline_engine handles eval_batch() and return_logits=True, then compare that path with Hugging Face CausalLM GenerationMixin and generate(). The issue names no files or tests; done would require an agreed integration that enables generate() during pipeline-parallel training evaluation without a separate generation class.