cschladetsch / cschladetsch/PyLocalEnsemble

VRAM arbiter

Open
#2 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1
Forks
0
No merged PRs in 30d

Description

Before image gen, signal llama.cpp to offload layers to CPU, generate, signal back. llama.cpp supports --n-gpu-layers via API at runtime

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.