lablup / lablup/backend.ai

Improve log handling

Open
#3,034 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

### Main idea

## References

- [MS teams thread](https://teams.microsoft.com/l/message/19:3c0acc0825024daeb2211f55c4e7f4aa@thread.skype/1730860992730?tenantId=13c6a44d-9b52-4b9e-aa34-0513ee7131f2&groupId=74ae2c4d-ec4d-4fdf-b2c2-f5041d1e8631&parentMessageId=1730860992730&teamName=devops&channelName=Backend.AI%20Talks&createdTime=1730860992730)
## Motivation

1. When db or etcd is terminated, the manager continues to retry the connection. However, in this case, an exception message is displayed instead of a log.
1. When the redis connection is terminated and the manager retries the connection, there is no log indicating when the connection is successfully restored. It is difficult to explicitly know the time of redis reconnection.
## Tasks

1. Ensure that an exception is logged when PostgreSQL or etcd connection retries fail.
1. When a redis request is retried and successfully completes, log the successful retry event as an info log.\* I think it would be better to run a ping request periodically to track the status of the redis server.

## Expected Results

- Consistent logging of both failure and recovery events, making it easier to track the issues and their resolutions.
### Alternative ideas

_No response_

### Anything else?

_No response_

JIRA Issue: BA-64

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.