abetlen / abetlen/llama-cpp-python

[FEAT]: TLS Certificate Support

Open
#1,768 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
10.6k
Forks
1.4k
PR merge metrics
PR metrics pending

Description

When trying to use self hosted endpoints running this software, I am encountering issues with other software such as "App Transport Security policy requires the use of a secure connection." My idea is to add functionality to pass in a certificate (e.g. from LetsEncrypt) to allow these other softwares to connect to the llama-cpp-python web server.

I have considered putting something in front of it like nginx or a reverse proxy (that my other self-hosted services use). I think it might be easier to enable TLS support in llama-cpp-python (via https://docs.python.org/3/library/ssl.html) until a few more years when hardware and docker etc. have better support for this technology for containerized solutions to make more sense.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.