pytorch / pytorch/executorch

Build qwen3 with qnn backend fails

Open
#10,895 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

module: qnn partner: qualcomm
Dominant language
Python
Stars
5k
Forks
1.2k
Avg merge
2d 10h
Merged PRs (30d)
581

Description

I'm trying to test Qwen3-0.6B model on Android with QNN backend, but struggling to export the model into a .pte file.

While exporting the model with the following command, I got an error [ERROR] [Qnn ExecuTorch]: Cannot Open QNN library libQnnHtp.so, with error: libc++.so.1: cannot open shared object file: No such file or directory. However, LD_LIBRARY_PATH is set properly and there's libQnnHtp.so file in the path.

python -m examples.models.llama.export_llama \
  --model qwen3-0_6b \
  --params examples/models/qwen3/0_6b_config.json \
  --use_kv_cache \
  --qnn \
  --pt2e_quantize qnn_16a4w \
  --metadata '{"get_bos_id": 151644, "get_eos_ids":[151645]}' \
  --output_name="qwen3-0_6b_qnn.pte" \
  --disable_dynamic_shape \
  --calibration_tasks wikitext --calibration_limit 1 --calibration_seq_length 128 --calibration_data "<|start_header_id|>system<|end_header_id|>\n\nYou are a funny chatbot.<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nCould you tell me about Facebook?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n" \
  --verbose

I used following commands to setup executorch with qnn backend and llama runner with qnn backend.

  • setup
cd $EXECUTORCH_ROOT
mkdir build-x86
cd build-x86
# Note that the below command might change.
# Please refer to the above build.sh for latest workable commands.
cmake .. \
  -DCMAKE_INSTALL_PREFIX=$PWD \
  -DEXECUTORCH_BUILD_QNN=ON \
  -DQNN_SDK_ROOT=${QNN_SDK_ROOT} \
  -DEXECUTORCH_BUILD_DEVTOOLS=ON \
  -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
  -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \
  -DEXECUTORCH_ENABLE_EVENT_TRACER=ON \
  -DPYTHON_EXECUTABLE=python3

# nproc is used to detect the number of available CPU.
# If it is not applicable, please feel free to use the number you want.
cmake --build $PWD --target "PyQnnManagerAdaptor" "PyQnnWrapperAdaptor" -j$(nproc)

# install Python APIs to correct import path
# The filename might vary depending on your Python and host version.
cp -f backends/qualcomm/PyQnnManagerAdaptor.cpython-310-x86_64-linux-gnu.so $EXECUTORCH_ROOT/backends/qualcomm/python
cp -f backends/qualcomm/PyQnnWrapperAdaptor.cpython-310-x86_64-linux-gnu.so $EXECUTORCH_ROOT/backends/qualcomm/python

# Workaround for fbs files in exir/_serialize
cp $EXECUTORCH_ROOT/schema/program.fbs $EXECUTORCH_ROOT/exir/_serialize/program.fbs
cp $EXECUTORCH_ROOT/schema/scalar_type.fbs $EXECUTORCH_ROOT/exir/_serialize/scalar_type.fbs
  • runner
cmake -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
    -DANDROID_ABI=arm64-v8a \
    -DANDROID_PLATFORM=android-23 \
    -DCMAKE_INSTALL_PREFIX=cmake-out-android \
    -DCMAKE_BUILD_TYPE=Release \
    -DEXECUTORCH_BUILD_EXTENSION_DATA_LOADER=ON \
    -DEXECUTORCH_BUILD_EXTENSION_MODULE=ON \
    -DEXECUTORCH_BUILD_EXTENSION_TENSOR=ON \
    -DEXECUTORCH_ENABLE_LOGGING=1 \
    -DPYTHON_EXECUTABLE=python \
    -DSUPPORT_REGEX_LOOKAHEAD=ON \
    -DEXECUTORCH_BUILD_QNN=ON \
    -DQNN_SDK_ROOT=$QNN_SDK_ROOT \
    -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON \
    -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON \
    -DEXECUTORCH_BUILD_KERNELS_CUSTOM=ON \
    -Bcmake-out-android .

cmake --build cmake-out-android -j16 --target install --config Release

cmake  -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
    -DANDROID_ABI=arm64-v8a \
    -DANDROID_PLATFORM=android-23 \
    -DCMAKE_INSTALL_PREFIX=cmake-out-android \
    -DCMAKE_BUILD_TYPE=Release \
    -DPYTHON_EXECUTABLE=python \
    -DSUPPORT_REGEX_LOOKAHEAD=ON \
    -DEXECUTORCH_BUILD_QNN=ON \
    -DQNN_SDK_ROOT=$QNN_SDK_ROOT \
    -DEXECUTORCH_BUILD_KERNELS_OPTIMIZED=ON \
    -DEXECUTORCH_BUILD_KERNELS_QUANTIZED=ON \
    -DEXECUTORCH_BUILD_KERNELS_CUSTOM=ON \
    -Bcmake-out-android/examples/models/llama \
    examples/models/llama

cmake --build cmake-out-android/examples/models/llama -j16 --config Release

Environments

  • executorch: main branch(b173722085b3f555d6ba4533d6bbaddfd7c71144)
  • development platform: Ubuntu 20.04
  • QNN version: 2.32.0.250228
  • Android NDK: r28b

cc @cccclai @winskuo-quic @shewu-quic @cbilgin

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure from examples/models/llama/export_llama using the QNN 2.32.0.250228 SDK and Android NDK r28b setup described in the issue. Trace the QNN library-loading path and verify the host export environment; done means the Qwen3 command produces qwen3-0_6b_qnn.pte without the missing libc++.so.1 error.

Written by the indexing model from the issue text.

Assessment

Tech stack
android, cmake, machine-learning, python
Domain
build-system, machine-learning, mobile
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.