huggingface / huggingface/huggingface-inference-toolkit

Fix `uvicorn` logging when in Vertex AI

Open
#75 1 comment 2 reactions 1 assignee Claimed by @alvarobartt View on GitHub
google
Dominant language
Python
Stars
97
Forks
28
Avg merge
6d 23m
Merged PRs (30d)
4

Description

## Description

At the moment, when using the latest wheel via the Hugging Face DCL in Google `us-docker.pkg.dev/deeplearning-platform-release/gcr.io/huggingface-pytorch-inference-cu121.2-2.transformers.4-41.ubuntu2204.py311`, the logs are not dumped within Google Cloud Logging, and that's due to the default `uvicorn` logging configuration.

> [!NOTE]
> **EDIT**: Those are indeed dumped, just that some logs take a while to appear within Google Cloud. But the logging messages on a per-request basis are missing, meaning that only failures are notified, so we could still try to add some logging messages for each incoming request.

Additionally, besides the default `uvicorn` logs on start up, GET/POST requests, etc. we should also add more `logging` messages within the inference interface to see what's going on when the inference is running to better debug the errors if any.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.