Lightning-AI / Lightning-AI/lightning-thunder

Proper logging of CI benchmarks

Open
#2,109 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
1.5k
Forks
121
PR merge metrics
No merged PRs in 30d

Description

## 🚀 Feature

It would be better for the CI benchmarks: https://github.com/Lightning-AI/lightning-thunder/blob/main/.github/run-benchmark-as-lit-jobs.py are logged somewhere easily observable and accessible.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with .github/run-benchmark-as-lit-jobs.py and trace how CI benchmark output is currently produced. Determine an accessible place for the benchmark logs and verify that a CI benchmark run makes its results observable there; the issue names no test or validation command.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, python
Domain
ci-cd, observability
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.