abetlen / abetlen/llama-cpp-python
Support Static File Dir/Root Dir in Server
- Dominant language
- Python
- Stars
- 10.6k
- Forks
- 1.4k
- PR merge metrics
- PR metrics pending
Description
**Is your feature request related to a problem? Please describe.**
The `server` in `llama_cpp_python` did not embed a UI for chat, which would be convenient in debugging/testing.
**Describe the solution you'd like**
While a lightweight web UI could be easily composed in a few hundred lines of vanilla HTML/js, modern browsers usually impose cross origin limitations for many APIs, and force the requests to be proxy-ed to align their origins with the web page, which increased the workload to set up the minimal environment.
However, the FastAPI which was imported in `server` module already provided support for `StaticFiles` which is perfect to host a lightweight web UI and other stuff. To achieve this, server settings should be augmented to include an argument for the path of static files or root, and the FastAPI `app` should `mount` it in `create_app()`.
**Describe alternatives you've considered**
N/A
**Additional context**
N/A
Contributor guide
Assessment
This issue has not been assessed yet.