huggingface / huggingface/huggingface-inference-toolkit
Fix `uvicorn` logging when in Vertex AI
- Dominant language
- Python
- Stars
- 97
- Forks
- 28
- Avg merge
- 6d 23m
- Merged PRs (30d)
- 4
Description
## Description
At the moment, when using the latest wheel via the Hugging Face DCL in Google `us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cu121.2-2.transformers.4-41.ubuntu2204.py311`, the logs are not dumped within Google Cloud Logging, and that's due to the default `uvicorn` logging configuration.
> [!NOTE]
> **EDIT**: Those are indeed dumped, just that some logs take a while to appear within Google Cloud. But the logging messages on a per-request basis are missing, meaning that only failures are notified, so we could still try to add some logging messages for each incoming request.
Additionally, besides the default `uvicorn` logs on start up, GET/POST requests, etc. we should also add more `logging` messages within the inference interface to see what's going on when the inference is running to better debug the errors if any.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.