linkedin / linkedin/Liger-Kernel

[RFC] Add Monitoring Interface for External Logging Backends

Open
#1,213 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
6.6k
Forks
603
Avg merge
1d 20h
Merged PRs (30d)
47

Description

### 🚀 The feature, motivation and pitch

Liger-Kernel provides a simple way to improve training throughput, but we need to manually implement logging to see these results while training.

A lightweight monitoring interface would:
1. Make Liger's benefits visible in real experiments
2. Help debug performance regressions across release versions

The integration can feature a common `liger_kernel.monitor` module with a backend agnostic interface that can enable adding external logging backends such as Weights and Biases, Trackio, Comet, etc. in a seamless manner. The external backends would be decoupled such that the backends would only be imported if they are installed in the user environment. This would also ensure that no external dependencies are added to the core dependency list.

Feel free to give feedback on this RFC on the scope of the API and its design.

### Alternatives

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

The RFC names no files, tests, or existing entry points. Start by reviewing the repository’s current logging and training integration points, then clarify the scope and API for a backend-agnostic monitor with optional external integrations. Done means an agreed design and implementation plan; specific validation is not defined here.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.