abetlen / abetlen/llama-cpp-python

Add support for auto setting n_gpu_layers from gguf and available vram size

Open
#1,456 0 comments 4 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
10.6k
Forks
1.4k
PR metrics pending

Description

This issue has no description.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.