Exported HF models contain SoftmaxCrossEntropyLoss node
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 346
- PR merge metrics
- No merged PRs in 30d
Description
I notice that using the torchscript exporter, ie using the --torchscript-onnx flag, all the exported HF models contain a SoftmaxCrossEntropyLoss node at the end that compares the model output with the true labels. Why is training-related ops showing up in the exported model and is there a way to disable such ops?
The export command I'm using is:
python pytorch/benchmarks/dynamo/huggingface.py --performance --amp -dcuda --output=/workspace/output/dynamo-onnx_huggingface_amp_inference_cuda_performance.csv --inference --use-eval-mode -n1 --torchscript-onnx --no-skip --dashboard -k <model_name>
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with pytorch/benchmarks/dynamo/huggingface.py and reproduce the export using the command shown with --torchscript-onnx and --inference. Trace where the exported model is built and determine why SoftmaxCrossEntropyLoss appears despite inference mode; done means explaining the behavior and identifying whether the command or exporter provides a way to omit it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 32/100