Unable to run llama_main with QNN Backend

Problem Description

I am trying to run llama_main with a QNN backend on my Android device (OnePlus 12, 24GB RAM). I followed the official tutorial https://pytorch.org/executorch/stable/llm/build-run-llama3-qualcomm-ai-engine-direct-backend.html to prepare the environment, upload the necessary files, and execute the command:

adb shell "su -c 'cd ${DEVICE_DIR} && export LD_LIBRARY_PATH=${DEVICE_DIR}:\$LD_LIBRARY_PATH && ./llama_main --model_path llama3_1B.pte --tokenizer_path tokenizer.model --prompt \"<|start_header_id|>system<|end_header_id|>\n\nYou are a funny chatbot.<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nCould you tell me about Facebook?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n\" --seq_len 128'"

However, the execution fails with the following error messages:

[INFO] [Qnn ExecuTorch]: create QNN Logger with log_level 2
[WARNING] [Qnn ExecuTorch]:  <W> Initializing HtpProvider

[WARNING] [Qnn ExecuTorch]:  <W> Function not called, PrepareLib isn't loaded!

[INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2
[INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE.
[ERROR] [Qnn ExecuTorch]:  <E> DspTransport.openSession qnn_open failed, 0x80000416

[ERROR] [Qnn ExecuTorch]:  <E> IDspTransport: Unknown rpc status 0x00000016

[ERROR] [Qnn ExecuTorch]:  <E> DspTransport failed,cannot open session, error 0xffffffff

[ERROR] [Qnn ExecuTorch]:  <E> Error from rpc transport. transportStatus = -1

[ERROR] [Qnn ExecuTorch]:  <E> Failed to retrieve skel build id: err: 1003

[ERROR] [Qnn ExecuTorch]:  <E> Failed to create transport for device, error: 4000

[ERROR] [Qnn ExecuTorch]:  <E> Failed to load skel, error: 4000

[ERROR] [Qnn ExecuTorch]:  <E> Transport layer setup failed: 14001

[ERROR] [Qnn ExecuTorch]:  <E> Failed to parse default platform info: 14001

[ERROR] [Qnn ExecuTorch]:  <E> Failed to load default platform info: 14001

[ERROR] [Qnn ExecuTorch]:  <E> Failed to parse platform config: 14001

[ERROR] [Qnn ExecuTorch]: Failed to create device_handle for Backend ID 6, error=14001
[WARNING] [Qnn ExecuTorch]:  <W> Backend 1 free cleanup called during process exit

[WARNING] [Qnn ExecuTorch]:  <W> qnnOpPackageManager: hexagon unload op package function pointer is nullptr!
...

The complete log is attached below.

Steps to Reproduce

1. Followed the to set up the environment.
2. Prepared the model and tokenizer:
   • Model: llama3_1B.pte
   • command:
     python -m examples.models.llama2.export_llama \ --checkpoint "../models/original/consolidated.00.pth" \ -p "../models/original/params.json" \ -kv \ --disable_dynamic_shape \ --qnn \ --pt2e_quantize qnn_16a4w \ -d fp32 \ --metadata '{"get_bos_id":128000, "get_eos_ids":[128009,128001]}' \ --output_name="llama3_1B.pte"
3. Uploaded necessary libraries to the device:
4. Executed the command mentioned above.

Environment

• Device: OnePlus 12 (24GB RAM)
• OS Version: Android 15
• ExecuTorch Version: 0.4.0a0
• QNN Version: 2.26.0
• Python Version: 3.10

Logs

Complete adb log output:

cc @cccclai @winskuo-quic @shewu-quic