abetlen / abetlen/llama-cpp-python

when depoly the llava-cpp-pyton server in k8s as a service , it can only answer questions about the first image

未关闭
#1,458 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
10.6k
派生
1.4k
PR 合并指标
PR 指标待抓取

描述

@abetlen Hello, when I use python -m llama_cpp.server deployed a llava13b service on the Kubernetes platform, I noticed an issue where only the first image could be correctly returned. When I switched to another image, the responses became completely confused. The only solution was to restart the service and resubmit the query. I suspect that the VRAM is not being released properly, and subsequent images are not being parsed correctly but rather seem to merge with previous images before being fed into the model, leading to highly inaccurate responses. This problem seems to still be related to VRAM. How should I resolve this issue?

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。