galaxyproject / galaxyproject/galaxy

Implement message categorization in logging and error reporting system

Open
#508 0 comments 0 reactions 0 assignees View on GitHub
feature-request friendliness/unfriendly kind/enhancement status/planning
Dominant language
Python
Stars
1.9k
Forks
1.2k
Avg merge
3d 51m
Merged PRs (30d)
148

Description

As discussed with @dannon and @erasche on IRC, it would be good to be able to tell apart logged messages based on who should be receiving them.
This is particularly useful when combined with Sentry which at the moment tends to exacerbate the criticality of a lot of the logged entries.

At the moment there are at least 3 kinds of "audience":
- Developers
- Core
- Tools
- Administrators
- Users
- API (bioblend, etc.)
- Web (Galaxy UI)

Administrators may not want to worry about errors they cannot fix themselves or that are of the responsibility of developers/users.
On the other end, API users need to be given information about what went wrong but that exact same information is insufficient (lacks context) for a developer or administrator to decide if it's relevant or not (for them).

Additionally, and this concerns mostly Sentry, in some cases the error message itself contains unique information (job_id, name of /tmp/ folder, PID, etc..) that prevents aggregation.
Every single message is logged as "new", resulting in excessive noise.

Extending the current model by providing extra information as metadata and including who should receive the message, would allow better filtering.

Some examples (note these are anecdotal examples, final implementation should hopefully be less verbose):

```
log.warning("Interface of module X is deprecated, use Y instead", developer=True)
log.critical("Failed to connect database", administrator=True, extra=db_connection_backtrace)
log.warning("Size of upload is too large", user=True, extra=size_of_upload) # user=True means Web and API
```

and a few complex cases:

```
log.error("Failed to run tool X", admin_msg="Cannot write to folder", admin=True, user=True, extra={"exception": tool_backtrace, "folder": output_folder})
log.exception(code_exception, developer=True, admin_msg="Internal exception while processing request of type X", admin=True, extra={"request": request})
```

The use of the `extra` attribute is to allow customizing the message and strip sensitive information as well as ensure repeated errors can be easily aggregated.

from @Unode on [trello](https://trello.com/c/ZJvO0FpY/2684-implement-message-categorization-in-logging-and-error-reporting-system)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.