abetlen / abetlen/llama-cpp-python

Setting verbose=False issues with threads

Đang mở
#1,874 1 bình luận 0 reaction 0 người được giao Xem trên GitHub
Ngôn ngữ chính
Python
Star
10.6k
Fork
1.4k
Chỉ số merge pull request
Chỉ số pull request đang chờ

Mô tả

# Prerequisites

Please answer the following questions for yourself before submitting an issue.

- [x] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [x] I carefully followed the [README.md](https://github.com/abetlen/llama-cpp-python/blob/main/README.md).
- [x] I [searched using keywords relevant to my issue](https://docs.github.com/en/issues/tracking-your-work-with-issues/filtering-and-searching-issues-and-pull-requests) to make sure that I am creating a new issue that is not already open (or closed).
- [x] I reviewed the [Discussions](https://github.com/abetlen/llama-cpp-python/discussions), and have a new bug or useful enhancement to share.

# Observed Behavior

Setting `verbose = False` when creating a `Llama` instance causes all program output to disappear while the model is running. This includes in particular `logging` output from other threads. In addition, if I create several `Llama` instances and use them in parallel from multiple threads, I sometimes receive some (but not all) output from one of the instances (I assume that's because another instance finished running first and restored the output for everyone). And sometimes output is _not_ restored when all instances have finished (probably because the last instance to finish restored output to a blocked state).

# Expected Behaviour

Setting `verbose = False` when creating a `Llama` instance should cause output from that instance to disappear. Other program output should still be visible (this includes `logging` output and output from other `Llama` instances).

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.