abetlen / abetlen/llama-cpp-python

Illegal instruction (core dumped) when trying to load model

Abierto
#839 4 comentarios 0 reacciones 0 asignados Ver en GitHub
bug
Lenguaje dominante
Python
Estrellas
10.6k
Forks
1.4k
Métricas de merge de PR
Métricas de PR pendientes

Descripción

# Prerequisites

Please answer the following questions for yourself before submitting an issue.

- [ x ] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [ x ] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [ x ] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [ x ] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.

# Expected Behavior
To load the model

Please provide a detailed written description of what you were trying to do, and what you expected `llama-cpp-python` to do.

# Current Behavior
when i try to load the model with `llm = Llama(model_path="./llama.cpp/models/llama-2-7b-chat.Q5_K_M.gguf")`
it response with: Illegal instruction (core dumped)

This is from my syslog:
kernel: [1728595.660950] traps: python3[213941] trap invalid opcode ip:7f4aa44a4e94 sp:7ffceec92e60 error:0 in libllama.so[7f4aa448a000+9f000]

# Environment and Context
`lscpu`

> Architecture: x86_64
> CPU op-mode(s): 32-bit, 64-bit
> Address sizes: 40 bits physical, 48 bits virtual
> Byte Order: Little Endian
> CPU(s): 8
> On-line CPU(s) list: 0-7
> Vendor ID: AuthenticAMD
> Model name: AMD Opteron 63xx class CPU
> CPU family: 21
> Model: 2
> Thread(s) per core: 1
> Core(s) per socket: 1
> Socket(s): 8
> Stepping: 0
> BogoMIPS: 5200.00
> Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx pdpe1gb rdtscp lm rep_good nopl cpuid extd_apicid tsc_known_freq pni pclmulqd
> q ssse3 fma cx16 sse4_1 sse4_2 x2apic popcnt aes xsave avx f16c hypervisor lahf_lm svm abm sse4a misalignsse 3dnowprefetch xop fma4 tbm vmmcall arat npt nrip_save
> Virtualization features:
> Virtualization: AMD-V
> Hypervisor vendor: KVM
> Virtualization type: full
> Caches (sum of all):
> L1d: 512 KiB (8 instances)
> L1i: 512 KiB (8 instances)
> L2: 4 MiB (8 instances)
> L3: 128 MiB (8 instances)
> NUMA:
> NUMA node(s): 1
> NUMA node0 CPU(s): 0-7
>

it is a Vritual with ubuntu 22.04

`$ uname -a`
Linux trying-to-train-llama2 5.15.0-46-generic #49-Ubuntu SMP Thu Aug 4 18:03:25 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

* SDK version, e.g. for Linux:

```
$ python3 --version
Python 3.10.12

$ make --version
GNU Make 4.3

$ g++ --version
g++ (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0

```

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.