Owner

abetlen

5 indexed repositories · View on GitHub

llama-cpp-python
Python bindings for llama.cpp

Python · 10616 stars
ggml-python
Python bindings for ggml

Python · 152 stars
sqlite3-bfsvtab-ext
A virtual table extension for breadth-first search queries in Sqlite3

C · 39 stars
pikchr-js
The Pikchr markup language for diagrams, running in your browser

C · 8 stars
simdinfo
Small C library for SIMD feature detection at runtime

C · 6 stars

Open All

613 open issues indexed

"tool_calls" not returning on native http request on a llama cpp server Open

abetlen/llama-cpp-python#1856 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Confusion regarding operation/terminology of speculative decoding and sampling Open

abetlen/llama-cpp-python#1865 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
It seems that LlamaSamplingContext is no longer in use. Open

abetlen/llama-cpp-python#1866 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
chatml-function-callling not adding tool description to the prompt. Open

abetlen/llama-cpp-python#1869 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Mambaforge breaks cuda builds, switch to miniforge? Open

abetlen/llama-cpp-python#1870 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Using a pre-built wheel currently requires specifying the right version, e.g. llama-cpp-python==0.3.4 Open

abetlen/llama-cpp-python#1872 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Setting verbose=False issues with threads Open

abetlen/llama-cpp-python#1874 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
issue tying to install on WSL Open

abetlen/llama-cpp-python#1876 · 4 comments · 3 reactions · 0 assignees · Indexed Sep 13, 2026
ROCM Build Target or Build Instructions are incorrect Open

abetlen/llama-cpp-python#1880 · 2 comments · 3 reactions · 0 assignees · Indexed Sep 13, 2026
How to create .so file for llama-cpp-python please help! Open

abetlen/llama-cpp-python#1881 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Feature request: add support for streaming tool use Open

abetlen/llama-cpp-python#1883 · 3 comments · 8 reactions · 0 assignees · Indexed Sep 13, 2026
MiniCPM-V 2.6 memory leak occurred !!! Open

abetlen/llama-cpp-python#1886 · 5 comments · 1 reaction · 0 assignees · Indexed Sep 13, 2026
The C compiler identification is unknown Open

abetlen/llama-cpp-python#1888 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Implementation doubt Open

abetlen/llama-cpp-python#1889 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Dependency issues with uvicorn Open

abetlen/llama-cpp-python#1892 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
cohere2 architecture issue Open

abetlen/llama-cpp-python#1893 · 0 comments · 1 reaction · 0 assignees · Indexed Sep 13, 2026
Does it support the gguf format model of Qwen2-VL-2B-Instruct Open

abetlen/llama-cpp-python#1895 · 7 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
llama_cpp_python-0.3.4-cp310-CU12 deepseekv3 /home/runner/work/llama-cpp-python/llama-cpp-python/vendor/llama.cpp/src/llama.cpp:5474: GGML_ASSERT(hparams.n_expert <= LLAMA_MAX_EXPERTS) failed Open

abetlen/llama-cpp-python#1896 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Generate answer from embedding vectors Open

abetlen/llama-cpp-python#1897 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Question - Batch Processing Open

abetlen/llama-cpp-python#1902 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
OSError: exception: access violation reading 0x0000000000000000 Open

abetlen/llama-cpp-python#1903 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Add minicpm-o and qwen2-vl to the list of supported multimodal models. Open

abetlen/llama-cpp-python#1904 · 11 comments · 2 reactions · 0 assignees · Indexed Sep 13, 2026
Wrong chat format for llava 1.5 Open

abetlen/llama-cpp-python#1905 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
openai API `n` argument is ignored Open

abetlen/llama-cpp-python#1906 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
openai API `max_completion_tokens` argument is ignored Open

abetlen/llama-cpp-python#1907 · 0 comments · 3 reactions · 0 assignees · Indexed Sep 13, 2026
pip install llama-cpp-python got stuck forever at "Configuring CMake" in docker Open

abetlen/llama-cpp-python#1908 · 4 comments · 2 reactions · 0 assignees · Indexed Sep 13, 2026
Automate upstream llama.cpp sync Open

abetlen/llama-cpp-python#1910 · 3 comments · 2 reactions · 0 assignees · Indexed Sep 13, 2026
RPC is broken due to change of interface in llama.cpp main repository (rpc : early register backend devices #11262) Open

abetlen/llama-cpp-python#1914 · 5 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Cuda Build Failed, Please take a look at this Open

abetlen/llama-cpp-python#1915 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
How to run CLIP model on GPU? Open

abetlen/llama-cpp-python#1916 · 2 comments · 5 reactions · 0 assignees · Indexed Sep 13, 2026
See `No CUDA toolset found` log after `-- CUDA Toolkit found.` log Open

abetlen/llama-cpp-python#1917 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Macos Metal Github Release for python 3.12 is broken Open

abetlen/llama-cpp-python#1918 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Distributed Inference Open

abetlen/llama-cpp-python#1921 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Unable to Build llama-cpp-python with Vulkan (Core Dump on Model Load) Open

abetlen/llama-cpp-python#1923 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Installing Llama cpp python on Debian with no git installed throws an error Open

abetlen/llama-cpp-python#1924 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Please release a cuda build for v0.3.5 Open

abetlen/llama-cpp-python#1925 · 9 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
`llama.cpp` transfer Open

abetlen/llama-cpp-python#1931 · 0 comments · 1 reaction · 0 assignees · Indexed Sep 13, 2026
Failed building wheel for llama-cpp-python Open

abetlen/llama-cpp-python#1932 · 6 comments · 1 reaction · 0 assignees · Indexed Sep 13, 2026
I cannot install this package with cuda12.4 Open

abetlen/llama-cpp-python#1933 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Using OpenBlas to accelerate has no effect？ Open

abetlen/llama-cpp-python#1935 · 0 comments · 1 reaction · 0 assignees · Indexed Sep 13, 2026
Getting seg faults intermittently prior to streaming generation Open

abetlen/llama-cpp-python#1936 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
Segmentation fault when converting embeddings into tensor Open

abetlen/llama-cpp-python#1937 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
No GPU usage, occupied VRAM but only CPU is working Open

abetlen/llama-cpp-python#1939 · 9 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
-- CUDA Toolkit found Open

abetlen/llama-cpp-python#1941 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
CMake build failed //Building wheel for llama-cpp-python (pyproject.toml) ... error Open

abetlen/llama-cpp-python#1943 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
CUDA Memory Allocation Failure and mlock Memory Lock Issue in llama-cpp-python Open

abetlen/llama-cpp-python#1944 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
OK!!! ggml_cuda_init: found 1 CUDA devices: Open

abetlen/llama-cpp-python#1945 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 13, 2026
The results generated are different from those produced by executing commands with the llama cpp library Open

abetlen/llama-cpp-python#1946 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
ERROR: Failed building wheel for llama-cpp-python Failed to build llama-cpp-python ERROR: Could not build wheels for llama-cpp-python which use PEP 517 and cannot be installed directly Open

abetlen/llama-cpp-python#1947 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026
successfully installed !! CUDA/v12.6 !! Visual Studio 2022!! Open

abetlen/llama-cpp-python#1948 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 13, 2026

abetlen

Get new issues in your inbox