huggingface / huggingface/optimum-executorch
albert fails to export
- Dominant language
- Python
- Stars
- 141
- Forks
- 48
- Avg merge
- 26m
- Merged PRs (30d)
- 2
Description
```
pytest -k test_albert_fill_mask -m run_slow
```
is failing during export stage
The log is as following
```
No ExecuTorch files were found for albert/albert-base-v2, setting `export=True` to convert the model to the ExecuTorch IR.
Some weights of the model checkpoint at albert/albert-base-v2 were not used when initializing AlbertForMaskedLM: ['albert.pooler.bias', 'albert.pooler.weight']
- This IS expected if you are initializing AlbertForMaskedLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing AlbertForMaskedLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] Error while creating guard:
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] Name: ''
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] Source: shape_env
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] Create Function: SHAPE_ENV
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] Guard Types: ['SHAPE_ENV', 'SHAPE_ENV', 'SHAPE_ENV', 'SHAPE_ENV', 'SHAPE_ENV', 'SHAPE_ENV', 'SHAPE_ENV', 'SHAPE_ENV', 'SHAPE_ENV', 'SHAPE_ENV']
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] Code List: ["L['input_ids'].size()[0] == 1", "L['input_ids'].stride()[0] == L['input_ids'].size()[1]", "L['input_ids'].stride()[1] == 1", "L['input_ids'].storage_offset() == 0", "L['attention_mask'].size()[0] == 1", "L['attention_mask'].size()[1] == L['input_ids'].size()[1]", "L['attention_mask'].stride()[0] == L['input_ids'].size()[1]", "L['attention_mask'].stride()[1] == 1", "L['attention_mask'].storage_offset() == 0", "2 <= L['input_ids'].size()[1] and L['input_ids'].size()[1] <= 512"]
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] Object Weakref: None
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] Guarded Class Weakref: None
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] Traceback (most recent call last):
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] File "/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/_inductor/cpp_builder.py", line 381, in _run_compile_cmd
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] subprocess.run(
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] File "/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/subprocess.py", line 571, in run
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] raise CalledProcessError(retcode, process.args,
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] subprocess.CalledProcessError: Command '['g++', '/tmp/torchinductor_chenlai/4u/c4uea23dlrkfyncufi7pzwcrqo7l5zqcw3pwyszry3geofmvrlc4.main.cpp', '-D', 'TORCH_INDUCTOR_CPP_WRAPPER', '-D', 'STANDALONE_TORCH_HEADER', '-D', 'C10_USING_CUSTOM_GENERATED_MACROS', '-shared', '-fPIC', '-O3', '-DNDEBUG', '-fno-trapping-math', '-funsafe-math-optimizations', '-ffinite-math-only', '-fno-signed-zeros', '-fno-math-errno', '-fexcess-precision=fast', '-fno-finite-math-only', '-fno-unsafe-math-optimizations', '-ffp-contract=off', '-fno-tree-loop-vectorize', '-march=native', '-Wall', '-std=c++17', '-Wno-unused-variable', '-Wno-unknown-pragmas', '-pedantic', '-fopenmp', '-I/home/chenlai/.conda/envs/optimum-executorch/include/python3.11', '-I/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/include', '-I/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/include/torch/csrc/api/include', '-o', '/tmp/torchinductor_chenlai/4u/c4uea23dlrkfyncufi7pzwcrqo7l5zqcw3pwyszry3geofmvrlc4.main.so', '-ltorch', '-ltorch_cpu', '-ltorch_python', '-lgomp', '-lc10', '-L/home/chenlai/.conda/envs/optimum-executorch/lib', '-L/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/lib']' returned non-zero exit status 1.
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0]
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] The above exception was the direct cause of the following exception:
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0]
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] Traceback (most recent call last):
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] File "/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/_guards.py", line 364, in create
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] return self.create_fn(builder, self)
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] File "/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/_dynamo/guards.py", line 2452, in SHAPE_ENV
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] clib = CppCodeCache.load(func_str)
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] File "/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/_inductor/codecache.py", line 2693, in load
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] return cls.load_async(*args, **kwargs)()
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] File "/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/_inductor/codecache.py", line 2676, in load_fn
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] result = worker_fn()
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] ^^^^^^^^^^^
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] File "/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/_inductor/codecache.py", line 2705, in _worker_compile_cpp
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] builder.build()
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] File "/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/_inductor/cpp_builder.py", line 1760, in build
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] run_compile_cmd(build_cmd, cwd=_build_tmp_dir)
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] File "/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/_inductor/cpp_builder.py", line 403, in run_compile_cmd
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] _run_compile_cmd(cmd_line, cwd)
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] File "/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/_inductor/cpp_builder.py", line 398, in _run_compile_cmd
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] raise exc.CppCompileError(cmd, output) from e
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] torch._inductor.exc.CppCompileError: C++ compile error
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0]
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] Command:
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] g++ /tmp/torchinductor_chenlai/4u/c4uea23dlrkfyncufi7pzwcrqo7l5zqcw3pwyszry3geofmvrlc4.main.cpp -D TORCH_INDUCTOR_CPP_WRAPPER -D STANDALONE_TORCH_HEADER -D C10_USING_CUSTOM_GENERATED_MACROS -shared -fPIC -O3 -DNDEBUG -fno-trapping-math -funsafe-math-optimizations -ffinite-math-only -fno-signed-zeros -fno-math-errno -fexcess-precision=fast -fno-finite-math-only -fno-unsafe-math-optimizations -ffp-contract=off -fno-tree-loop-vectorize -march=native -Wall -std=c++17 -Wno-unused-variable -Wno-unknown-pragmas -pedantic -fopenmp -I/home/chenlai/.conda/envs/optimum-executorch/include/python3.11 -I/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/include -I/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/include/torch/csrc/api/include -o /tmp/torchinductor_chenlai/4u/c4uea23dlrkfyncufi7pzwcrqo7l5zqcw3pwyszry3geofmvrlc4.main.so -ltorch -ltorch_cpu -ltorch_python -lgomp -lc10 -L/home/chenlai/.conda/envs/optimum-executorch/lib -L/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/lib
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0]
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] Output:
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] In file included from /tmp/libcxx-14.0.0/include/__bits:13,
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] from /tmp/libcxx-14.0.0/include/algorithm:662,
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] from /tmp/torchinductor_chenlai/4u/c4uea23dlrkfyncufi7pzwcrqo7l5zqcw3pwyszry3geofmvrlc4.main.cpp:2:
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] /tmp/libcxx-14.0.0/include/__config:13:10: fatal error: __config_site: No such file or directory
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] 13 | #include <__config_site>
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] | ^~~~~~~~~~~~~~~
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0] compilation terminated.
E0805 09:10:47.131000 2408011 site-packages/torch/_guards.py:366] [0/0]
E0805 09:10:47.138000 2408011 site-packages/torch/_guards.py:368] [0/0] Created at:
E0805 09:10:47.138000 2408011 site-packages/torch/_guards.py:368] [0/0] File "/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/_dynamo/convert_frame.py", line 795, in transform
E0805 09:10:47.138000 2408011 site-packages/torch/_guards.py:368] [0/0] tracer = InstructionTranslator(
E0805 09:10:47.138000 2408011 site-packages/torch/_guards.py:368] [0/0] File "/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/_dynamo/symbolic_convert.py", line 3356, in __init__
E0805 09:10:47.138000 2408011 site-packages/torch/_guards.py:368] [0/0] output=OutputGraph(
E0805 09:10:47.138000 2408011 site-packages/torch/_guards.py:368] [0/0] File "/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/_dynamo/output_graph.py", line 470, in __init__
E0805 09:10:47.138000 2408011 site-packages/torch/_guards.py:368] [0/0] self.init_ambient_guards()
E0805 09:10:47.138000 2408011 site-packages/torch/_guards.py:368] [0/0] File "/home/chenlai/.conda/envs/optimum-executorch/lib/python3.11/site-packages/torch/_dynamo/output_graph.py", line 623, in init_ambient_guards
E0805 09:10:47.138000 2408011 site-packages/torch/_guards.py:368] [0/0] self.guards.add(ShapeEnvSource().make_guard(GuardBuilder.SHAPE_ENV))
--------------------------------------------------------------------------- Captured log call ----------------------------------------------------------------------------
WARNING coremltools:__init__.py:75 scikit-learn version 1.7.1 is not supported. Minimum required version: 0.17. Maximum required version: 1.5.1. Disabling scikit-learn conversion API.
WARNING coremltools:__init__.py:35 Torch version 2.9.0.dev20250801+cpu has not been tested with coremltools. You may run into unexpected errors. Torch 2.5.0 is the most recent version that has been tested.
WARNING coremltools:compute_device.py:14 Failed to load _MLModelProxy: No module named 'coremltools.libcoremlpython'
WARNING coremltools:compute_device.py:20 Failed to load _MLCPUComputeDeviceProxy: No module named 'coremltools.libcoremlpython'
WARNING coremltools:compute_device.py:26 Failed to load _MLGPUComputeDeviceProxy: No module named 'coremltools.libcoremlpython'
WARNING coremltools:compute_device.py:32 Failed to load _MLNeuralEngineComputeDeviceProxy: No module named 'coremltools.libcoremlpython'
WARNING coremltools:compute_plan.py:22 Failed to load _MLModelProxy: No module named 'coremltools.libcoremlpython'
WARNING coremltools:compute_plan.py:28 Failed to load _MLComputePlanProxy: No module named 'coremltools.libcoremlpython'
WARNING coremltools:model.py:57 Failed to load _MLModelProxy: No module named 'coremltools.libcoremlpython'
WARNING coremltools:model.py:64 Failed to load _MLModelAssetProxy: No module named 'coremltools.libcoremlpython'
```
Contributor guide
Assessment
This issue has not been assessed yet.