pytorch / pytorch/executorch

[QNN] Context creation fails with SMMU fastrpc mmap error (err 1002) when loading large weight buffer (~467MB)

Open
#18,862 1 comment 0 reactions 1 assignee View on GitHub

@chenweng-quic is already working on this.

Since Apr 14, 2026.

module: qnn
Dominant language
Python
Stars
5k
Forks
1.2k
Avg merge
2d 10h
Merged PRs (30d)
581

Description

🐛 Describe the bug

I have successfully run InternVL-1B inference on SA8295 #18410 . When attempting to run InternVL-2B, I encountered this memory mapping failure error. The following are some of the changes made for InternVL-2B ，I increased the number of model shards to 24 and used the same quantization recipe as internvl-1b. ：

diff --git a/examples/qualcomm/oss_scripts/llama/__init__.py b/examples/qualcomm/oss_scripts/llama/__init__.py
index 0fbb18fb6d..003c226343 100644
--- a/examples/qualcomm/oss_scripts/llama/__init__.py
+++ b/examples/qualcomm/oss_scripts/llama/__init__.py
@@ -66,6 +66,7 @@ from executorch.examples.qualcomm.oss_scripts.llama.static_llm_quant_recipe impo
     GLM_1_5B_InstructQuantRecipe,
     Granite_3_3_2B_InstructQuantRecipe,
     InternVL3_1B_QuantRecipe,
+    InternVL3_2B_QuantRecipe,
     Llama3_1BQuantRecipe,
     Llama3_3BQuantRecipe,
     LlamaStories110MQuantRecipe,
@@ -90,6 +91,7 @@ LLM_VARIANT_ARCHS: Dict[str, LlamaModel] = {
     "gemma3-1b": MultiScopeAwareLlamaModel,
     "smolvlm_500m_instruct": LlamaModelWithoutEmbedding,
     "internvl3_1b": LlamaModelWithoutEmbedding,
+    "internvl3_2b": LlamaModelWithoutEmbedding,
     "gemma2-2b": MultiScopeAwareLlamaModel,
 }
 
 
+@register_llm_model(
+    "internvl3_2b",
+    vision_encoder=InternVL3Encoder,
+)
+@dataclass(init=False, frozen=True)
+class InternVL3_2B(LLMModelConfig):
+    repo_id: str = "OpenGVLab/InternVL3-2B-hf"
+    params_path: str = os.path.join(
+        BASE_DIR, "../../../models/internvl3/2b_config.json"
+    )
+    convert_weights = convert_internvl3_weights
+    transform_weight = False
+    instruct_model = True
+    num_sharding = 24
+    masked_softmax = True
+    seq_mse_candidates = 0
+    r1 = False
+    r2 = False
+    r3 = False
+    quant_recipe = InternVL3_2B_QuantRecipe
+
+
 @register_llm_model(
     "smolvlm_500m_instruct",
     vision_encoder=SmolVLMEncoder,

class InternVL3_1B_QuantRecipe(StaticLLMQuantRecipe):
    default_quant_dtype = QuantDtype.use_16a8w

    def __init__(self, verbose: bool = False):
        super().__init__()

        self.recipe = QuantRecipe(
            self.default_quant_dtype,
            False,
            act_observer=MinMaxObserver,
            granularity=QuantGranularity.PER_TENSOR,
            verbose=verbose,
        ).add_node_target(
            {
                torch.ops.aten.conv2d.default,
            },
            QuantDtype.use_16a8w,
            False,
            act_observer=MinMaxObserver,
            granularity=QuantGranularity.PER_CHANNEL,
        )
        self.recipe.custom_quant_annotations.extend([custom_annotation_16a4w_layer_norm,custom_annotate_matmul_16a8w, annotate_kv_8bit])


class InternVL3_2B_QuantRecipe(StaticLLMQuantRecipe):
      default_quant_dtype = QuantDtype.use_16a8w

      def __init__(self, verbose: bool = False):
        super().__init__()

        self.recipe = QuantRecipe(
            self.default_quant_dtype,
            False,
            act_observer=MinMaxObserver,
            granularity=QuantGranularity.PER_TENSOR,
            verbose=verbose,
        ).add_node_target(
            {
                torch.ops.aten.conv2d.default,
            },
            QuantDtype.use_16a8w,
            False,
            act_observer=MinMaxObserver,
            granularity=QuantGranularity.PER_CHANNEL,
        )
        self.recipe.custom_quant_annotations.extend([custom_annotation_16a4w_layer_norm,custom_annotate_matmul_16a8w, annotate_kv_8bit])
error log
Click to expand log I 00:00:00.001091 executorch:qnn_multimodal_runner.cpp:247] Load Encoder: vision_encoder_qnn.pte I 00:00:00.001147 executorch:qnn_multimodal_runner.cpp:254] Load Token Embedding: tok_embedding_qnn.pte I 00:00:00.001370 executorch:qnn_multimodal_runner.cpp:261] Load Text Decoder: hybrid_llama_qnn.pte I 00:00:00.001647 executorch:qnn_multimodal_runner.cpp:139] Starting multimodal runner I 00:00:00.001664 executorch:multimodal_runner.cpp:144] creating runner: tokenizer_path=tokenizer.json I 00:00:00.001668 executorch:multimodal_runner.cpp:145] eval mode=1 I tokenizers:normalizer.cpp:102] Using NFC normalizer. Please notice that our implementation may not handle all edge cases. WARNING: All log messages before absl::InitializeLog() is called are written to STDERR E0000 00:00:1776137332.156216 19163 re2.cc:237] Error parsing '((?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\r\n\p{L}\p{N}]?\p{L}+|\p{N}| ?[^\s\p{L}\p{N}]+[\r\n]*|\s*[\r\n]+|\s...': invalid perl operator: (?! I tokenizers:re2_regex.cpp:27] Re2 failed to compile regex: ((?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\r\n\p{L}\p{N}]?\p{L}+|\p{N}| ?[^\s\p{L}\p{N}]+[\r\n]*|\s*[\r\n]+|\s+(?!\S)|\s+), error: invalid perl operator: (?! This may be ok if a fallback regex is used. I tokenizers:regex_lookahead.cpp:27] Creating PCRE2 regex I 00:00:01.450881 executorch:llm_runner_helper.cpp:54] Loaded json tokenizer [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Creating new backend bundle. [INFO] [Qnn ExecuTorch]: create QNN Logger with log_level 1 [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached delegate handle for current method: kv_forward I 00:00:04.206583 executorch:multimodal_runner.cpp:219] Reading metadata from model [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [INFO] [Qnn ExecuTorch]: Running level=1 optimization. [INFO] [Qnn ExecuTorch]: Deserializing processed data using QnnContextCustomProtocol [INFO] [Qnn ExecuTorch]: Use cached backend bundle for current backend: kHtpBackend [INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2 [INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE. [INFO] [Qnn ExecuTorch]: QnnContextCustomProtocol expected magic number: 0x5678abcd but get: 0x2000000 [ERROR] [Qnn ExecuTorch]: QnnDsp fastrpc memory map for fd: 363 with length: 467664896 failed with error: 0x1

[ERROR] [Qnn ExecuTorch]: QnnDsp fastrpc memory map error reporting failed

[ERROR] [Qnn ExecuTorch]: QnnDsp Mapping buffer fd 363 to FastRpc failed on domain 3

[ERROR] [Qnn ExecuTorch]: QnnDsp SharedMemoryMod failed to Map Buffer to SMMU for domain 0

[ERROR] [Qnn ExecuTorch]: QnnDsp Failed to map buffer of size 467664896

[ERROR] [Qnn ExecuTorch]: QnnDsp Failed to map weights buffer to device!

[ERROR] [Qnn ExecuTorch]: QnnDsp Could not map shared weights buffer!

[ERROR] [Qnn ExecuTorch]: QnnDsp Failed to map shared weights for contextId 37 on deviceId 0, coreId 0, pdId 0, err: 1002

[ERROR] [Qnn ExecuTorch]: QnnDsp Context create from binary failed for deviceId 0 coreId 0 pdId 0 for context 25, err 1002

[ERROR] [Qnn ExecuTorch]: QnnDsp Error from rpc transport res[0]:8003

[ERROR] [Qnn ExecuTorch]: QnnDsp Context 37 failed on pd 0

[ERROR] [Qnn ExecuTorch]: QnnDsp DspTransport.createUnsignedPD unable to load unsigned PD

[ERROR] [Qnn ExecuTorch]: QnnDsp DspTransport.createUnsignedPD failed, 0x00000006

[ERROR] [Qnn ExecuTorch]: QnnDsp DspTransport.getHandle failed, error 0x00000006

[ERROR] [Qnn ExecuTorch]: QnnDsp createDspTransportInstance failed to config transport object

[ERROR] [Qnn ExecuTorch]: QnnDsp error in creation of transport instance

[ERROR] [Qnn ExecuTorch]: QnnDsp Failed to create a new transport session for deviceId 0, coreId 0, pdId 2: err: 1002

[ERROR] [Qnn ExecuTorch]: QnnDsp Error in creating transport session for deviceId 0, coreId 0, pdId 2, err: 1002

[ERROR] [Qnn ExecuTorch]: QnnDsp Error in creating transport session for deviceId 0, coreId 0, pdId 2, err: 1002

[ERROR] [Qnn ExecuTorch]: QnnDsp DspTransport.createUnsignedPD unable to load unsigned PD

[ERROR] [Qnn ExecuTorch]: QnnDsp DspTransport.createUnsignedPD failed, 0x00000006

[ERROR] [Qnn ExecuTorch]: QnnDsp DspTransport.getHandle failed, error 0x00000006

[ERROR] [Qnn ExecuTorch]: QnnDsp createDspTransportInstance failed to config transport object

[ERROR] [Qnn ExecuTorch]: QnnDsp error in creation of transport instance

[ERROR] [Qnn ExecuTorch]: QnnDsp Failed to create a new transport session for deviceId 0, coreId 0, pdId 3: err: 1002

[ERROR] [Qnn ExecuTorch]: QnnDsp Error in creating transport session for deviceId 0, coreId 0, pdId 3, err: 1002

[ERROR] [Qnn ExecuTorch]: QnnDsp Error in creating transport session for deviceId 0, coreId 0, pdId 3, err: 1002

[ERROR] [Qnn ExecuTorch]: QnnDsp DspTransport.createUnsignedPD unable to load unsigned PD

[ERROR] [Qnn ExecuTorch]: QnnDsp DspTransport.createUnsignedPD failed, 0x00000006

[ERROR] [Qnn ExecuTorch]: QnnDsp DspTransport.getHandle failed, error 0x00000006

[ERROR] [Qnn ExecuTorch]: QnnDsp createDspTransportInstance failed to config transport object

[ERROR] [Qnn ExecuTorch]: QnnDsp error in creation of transport instance

[ERROR] [Qnn ExecuTorch]: QnnDsp Failed to create a new transport session for deviceId 0, coreId 0, pdId 1: err: 1002

[ERROR] [Qnn ExecuTorch]: QnnDsp Error in creating transport session for deviceId 0, coreId 0, pdId 1, err: 1002

[ERROR] [Qnn ExecuTorch]: QnnDsp Error in creating transport session for deviceId 0, coreId 0, pdId 1, err: 1002

[ERROR] [Qnn ExecuTorch]: QnnDsp Failed to find available PD for contextId 37 on deviceId 0 coreId 0with context size estimate 486802688

[ERROR] [Qnn ExecuTorch]: QnnDsp context create from binary failed on contextId 37, err = 1002

[ERROR] [Qnn ExecuTorch]: QnnDsp context create from binary failed for contextId 37, err = 1002

[ERROR] [Qnn ExecuTorch]: QnnDsp Fail to create context from binary with err 1002

[ERROR] [Qnn ExecuTorch]: QnnDsp Failed to create context from binary with err 0x3ea

[ERROR] [Qnn ExecuTorch]: Can't create context from binary. Error 1002.
E 00:00:06.622743 executorch:QnnManager.cpp:268] Fail to configure Qnn context
E 00:00:06.622774 executorch:QnnExecuTorchBackend.cpp:99] Fail to initialize Qnn Manager
E 00:00:06.622847 executorch:method.cpp:114] Init failed for backend QnnBackend: 0x1
F 00:00:06.714608 executorch:result.h:170] In function CheckOk(), assert failed: hasValue_
Aborted

looking forward to your help
cc @cccclai @winskuo-quic @shewu-quic @haowhsu-quic @DannyYuyang-quic @cbilgin @abhinaykukkadapu

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.