lablup / lablup/backend.ai-webui

Structured training-metrics file convention (metrics.jsonl) for session metric charts

Open
#9,001 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
133
Forks
81
Avg merge
1d 11h
Merged PRs (30d)
344

Description

## Problem

Phase 1 (log-tail regex parsing) covers common trainer formats but is inherently fragile — custom print formats or redirected output break it.

## Proposal (Phase 2 of the training-metrics series)

Define an **opt-in structured metrics convention**: a session writes `.logs/metrics.jsonl` (one JSON object per line: `{"step": n, "name": "loss", "value": x, "ts": ...`}), and the WebUI reads it through the existing vfolder file API to feed the same Metrics tab chart introduced in Phase 1.

- Structured source takes precedence over log parsing when both exist.
- Document the convention in the user manual, including one-line integration snippets (HF Trainer callback, Lightning logger, plain Python helper).
- Consider TensorBoard event-file reading as a stretch goal (heavier parsing; separate decision).

## Acceptance criteria

- A session writing `metrics.jsonl` per the convention shows charts regardless of what its stdout looks like.
- Precedence (structured > parsed) is applied and covered by tests.
- User-manual page exists for the convention with copy-pastable snippets.

## Note

A proper backend-collected metrics push API is the long-term answer; that requires a Backend.AI core (BA) counterpart issue and is out of scope here — this phase is TODO(needs-backend)-free by design.

JIRA Issue: FR-3649

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the existing Metrics tab and the Phase 1 log-tail parser, then trace the vfolder file API used to read session files. Add tests for structured metrics and structured-over-parsed precedence, and update the user manual with the convention and the three requested integration snippets. Done means metrics.jsonl produces charts independently of stdout and the documented tests pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
documentation, frontend, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.