Export a forward-only PTE model (with forward() returns loss + mutable params) for gradient-free finetuning

🚀 The feature, motivation and pitch

Hi team,

I’m working on a project that needs to finetune a PTE model. In the "llm_fine_tuning" example, the model is exported with a combined forward+backward graph via _export_forward_backward():

https://github.com/pytorch/executorch/blob/7ce78c00367019ecffec9b979efda59ee6715ec6/examples/llm_pte_finetuning/model_loading_lib.py#L75

When I run forward() on the exported PTE model, the outputs are loss + gradients + parameters (with parameters being mutable), per the runner code:

https://github.com/pytorch/executorch/blob/7ce78c00367019ecffec9b979efda59ee6715ec6/examples/llm_pte_finetuning/runner.py#L85-L86

https://github.com/pytorch/executorch/blob/7ce78c00367019ecffec9b979efda59ee6715ec6/examples/llm_pte_finetuning/runner.py#L114-L120

What I am trying to do

I would like to export a forward-only compute graph where the parameters remain mutable, and the forward() return format is simply loss + parameters (i.e., no gradients)

This would let me apply gradient-free optimization methods for finetuning, which could reduce runtime and memory by skipping the backward graph entirely.

Alternatives

• Is there a recommended or supported way to export a forward-only PTE model while keeping parameters mutable?

• If so, is there an API flag or alternative export path (e.g., an _export_forward() variant) I should use?

• If not currently supported, would this be a feasible feature request? Any pointers on where to start would be greatly appreciated.

Thanks for your time and for the great project!

Additional context

No response

RFC (Optional)

No response

Start with examples/llm_pte_finetuning/model_loading_lib.py at _export_forward_backward() and the corresponding calls in examples/llm_pte_finetuning/runner.py. Trace the available export paths and parameter mutability behavior; done when a forward-only PTE export path is identified or specified that returns loss and mutable parameters without gradients.