onnx / onnx/optimizer

Optimizer return an empty model

Open
#144 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
834
Forks
109
Avg merge
6h 55m
Merged PRs (30d)
2

Description

Hello,

I have an issue using this command line python -m onnxoptimizer "model.onnx" "model_opti.onnx"

The script fail at onnx.checker.check_model(output_file) in file onnxoptimizer_main.py line 85
with the error : onnx.onnx_cpp2py_export.checker.ValidationError: model with IR version >= 3 must specify opset_import for ONNX

When i check what's in the model file, i only find
ir_version: 8
producer_name: "pytorch"
producer_version: "2.1.0"
graph {
}

Now more context

i took the model "togethercomputer/LLaMA-2-7B-32K" in huggingface
convert with the following commands

from transformers import AutoTokenizer, LlamaForCausalLM

CACHE_DIR = r".\cache_dir"

tokenizer = AutoTokenizer.from_pretrained("togethercomputer/LLaMA-2-7B-32K", cache_dir=CACHE_DIR)
model = LlamaForCausalLM.from_pretrained("togethercomputer/LLaMA-2-7B-32K", cache_dir=CACHE_DIR, use_safetensors = False)

prompt = "test"
inputs = tokenizer(prompt, return_tensors="pt")

input_names = ["input_ids"]
output_names = ["output"]
torch.onnx.export(model, inputs.input_ids, r'.\model\llama2_32k_with_weight.onnx',export_params=True, input_names=input_names, output_names=output_names, dynamic_axes={'input_ids' : {1 : 'context_length'}, 'output' : {1 : 'context_length'}})

`from transformers import AutoTokenizer, LlamaForCausalLM

CACHE_DIR = r".\cache_dir"

tokenizer = AutoTokenizer.from_pretrained("togethercomputer/LLaMA-2-7B-32K", cache_dir=CACHE_DIR)
model = LlamaForCausalLM.from_pretrained("togethercomputer/LLaMA-2-7B-32K", cache_dir=CACHE_DIR, use_safetensors = False)

prompt = "test"
inputs = tokenizer(prompt, return_tensors="pt")

input_names = ["input_ids"]
output_names = ["output"]
torch.onnx.export(model, inputs.input_ids, r'.\model\llama2_32k_with_weight.onnx',export_params=True, input_names=input_names, output_names=output_names, dynamic_axes={'input_ids' : {1 : 'context_length'}, 'output' : {1 : 'context_length'}})`from transformers import AutoTokenizer, LlamaForCausalLM

CACHE_DIR = r".\cache_dir"

tokenizer = AutoTokenizer.from_pretrained("togethercomputer/LLaMA-2-7B-32K", cache_dir=CACHE_DIR)
model = LlamaForCausalLM.from_pretrained("togethercomputer/LLaMA-2-7B-32K", cache_dir=CACHE_DIR, use_safetensors = False)

prompt = "test"
inputs = tokenizer(prompt, return_tensors="pt")

input_names = ["input_ids"]
output_names = ["output"]
torch.onnx.export(model, inputs.input_ids, r'.\model\llama2_32k_with_weight.onnx',export_params=True, input_names=input_names, output_names=output_names, dynamic_axes={'input_ids' : {1 : 'context_length'}, 'output' : {1 : 'context_length'}})

After that i use :

python -m onnxoptimizer "llama2_32k_with_weight.onnx" "model_opti.onnx"

And it fail

I tried to check where the optimizer fail exactly

in onnxoptimizer\init

model_str = model.SerializeToString()
length of model_str is 26988224572

optimized_model_str =C.optimize(model_str, passes)
length of model_str is 20

If someone have an idea
Thank in advance,
I use your package for a long time and it's the first time i encounter a problem, love your work

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with the reported python -m onnxoptimizer command and inspect onnxoptimizer_main.py line 85, then trace the call in onnxoptimizer/init through C.optimize. Compare the serialized model and optimized output sizes; done means the optimizer returns a valid non-empty ONNX model with the required opset information.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.