[QCOM] [Llama] Gibberish Result on Llama 3.2 1B

🐛 Describe the bug

I am getting gibberish results on inference.
Model: Llama 3.2 1B
Phone: Galaxy s24 Ultra
SDK: 2.28
NDK: 26d
Specification to generate .pte file

python -m examples.models.llama.export_llama --checkpoint "/home/qaaiml-shreshth/executorch_qualcomm/dependencies/consolidated.00.pth" -t "/home/qaaiml-shreshth/executorch_qualcomm/dependencies/tokenizer.model" -p "/home/qaaiml-shreshth/executorch_qualcomm/dependencies/params.json" -kv --disable_dynamic_shape --qnn --pt2e_quantize qnn_16a4w -d fp32 --metadata '{"get_bos_id":128000, "get_eos_ids":[128009, 128001]}' --output_name="llama3.2-1B.pte" --calibration_data "<|start_header_id|>system<|end_header_id|>\n\nYou are a funny chatbot.<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nCould you tell me about Facebook?<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n

Versions

-null

cc @cccclai @winskuo-quic @shewu-quic @cbilgin

Start with examples/models/llama/export_llama and reproduce the supplied Llama 3.2 1B export on the Galaxy S24 Ultra using the listed SDK, NDK, QNN, and quantization options. Compare the inference output with the expected response and investigate the export or device configuration until the result is no longer gibberish.