Optimizer return an empty model
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 834
- Forks
- 109
- Avg merge
- 6h 55m
- Merged PRs (30d)
- 2
Description
Hello,
I have an issue using this command line python -m onnxoptimizer "model.onnx" "model_opti.onnx"
The script fail at onnx.checker.check_model(output_file) in file onnxoptimizer_main.py line 85
with the error : onnx.onnx_cpp2py_export.checker.ValidationError: model with IR version >= 3 must specify opset_import for ONNX
When i check what's in the model file, i only find
ir_version: 8
producer_name: "pytorch"
producer_version: "2.1.0"
graph {
}
Now more context
i took the model "togethercomputer/LLaMA-2-7B-32K" in huggingface
convert with the following commands
from transformers import AutoTokenizer, LlamaForCausalLM
CACHE_DIR = r".\cache_dir"
tokenizer = AutoTokenizer.from_pretrained("togethercomputer/LLaMA-2-7B-32K", cache_dir=CACHE_DIR)
model = LlamaForCausalLM.from_pretrained("togethercomputer/LLaMA-2-7B-32K", cache_dir=CACHE_DIR, use_safetensors = False)
prompt = "test"
inputs = tokenizer(prompt, return_tensors="pt")
input_names = ["input_ids"]
output_names = ["output"]
torch.onnx.export(model, inputs.input_ids, r'.\model\llama2_32k_with_weight.onnx',export_params=True, input_names=input_names, output_names=output_names, dynamic_axes={'input_ids' : {1 : 'context_length'}, 'output' : {1 : 'context_length'}})
`from transformers import AutoTokenizer, LlamaForCausalLM
CACHE_DIR = r".\cache_dir"
tokenizer = AutoTokenizer.from_pretrained("togethercomputer/LLaMA-2-7B-32K", cache_dir=CACHE_DIR)
model = LlamaForCausalLM.from_pretrained("togethercomputer/LLaMA-2-7B-32K", cache_dir=CACHE_DIR, use_safetensors = False)
prompt = "test"
inputs = tokenizer(prompt, return_tensors="pt")
input_names = ["input_ids"]
output_names = ["output"]
torch.onnx.export(model, inputs.input_ids, r'.\model\llama2_32k_with_weight.onnx',export_params=True, input_names=input_names, output_names=output_names, dynamic_axes={'input_ids' : {1 : 'context_length'}, 'output' : {1 : 'context_length'}})`from transformers import AutoTokenizer, LlamaForCausalLM
CACHE_DIR = r".\cache_dir"
tokenizer = AutoTokenizer.from_pretrained("togethercomputer/LLaMA-2-7B-32K", cache_dir=CACHE_DIR)
model = LlamaForCausalLM.from_pretrained("togethercomputer/LLaMA-2-7B-32K", cache_dir=CACHE_DIR, use_safetensors = False)
prompt = "test"
inputs = tokenizer(prompt, return_tensors="pt")
input_names = ["input_ids"]
output_names = ["output"]
torch.onnx.export(model, inputs.input_ids, r'.\model\llama2_32k_with_weight.onnx',export_params=True, input_names=input_names, output_names=output_names, dynamic_axes={'input_ids' : {1 : 'context_length'}, 'output' : {1 : 'context_length'}})
After that i use :
python -m onnxoptimizer "llama2_32k_with_weight.onnx" "model_opti.onnx"
And it fail
I tried to check where the optimizer fail exactly
in onnxoptimizer\init
model_str = model.SerializeToString()
length of model_str is 26988224572
optimized_model_str =C.optimize(model_str, passes)
length of model_str is 20
If someone have an idea
Thank in advance,
I use your package for a long time and it's the first time i encounter a problem, love your work
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with the reported python -m onnxoptimizer command and inspect onnxoptimizer_main.py line 85, then trace the call in onnxoptimizer/init through C.optimize. Compare the serialized model and optimized output sizes; done means the optimizer returns a valid non-empty ONNX model with the required opset information.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 28/100