lablup / lablup/backend.ai-webui
Structured training-metrics file convention (metrics.jsonl) for session metric charts
- Dominant language
- TypeScript
- Stars
- 133
- Forks
- 81
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 344
Description
## Problem
Phase 1 (log-tail regex parsing) covers common trainer formats but is inherently fragile — custom print formats or redirected output break it.
## Proposal (Phase 2 of the training-metrics series)
Define an **opt-in structured metrics convention**: a session writes `.logs/metrics.jsonl` (one JSON object per line: `{"step": n, "name": "loss", "value": x, "ts": ...`}), and the WebUI reads it through the existing vfolder file API to feed the same Metrics tab chart introduced in Phase 1.
- Structured source takes precedence over log parsing when both exist.
- Document the convention in the user manual, including one-line integration snippets (HF Trainer callback, Lightning logger, plain Python helper).
- Consider TensorBoard event-file reading as a stretch goal (heavier parsing; separate decision).
## Acceptance criteria
- A session writing `metrics.jsonl` per the convention shows charts regardless of what its stdout looks like.
- Precedence (structured > parsed) is applied and covered by tests.
- User-manual page exists for the convention with copy-pastable snippets.
## Note
A proper backend-collected metrics push API is the long-term answer; that requires a Backend.AI core (BA) counterpart issue and is out of scope here — this phase is TODO(needs-backend)-free by design.
JIRA Issue: FR-3649
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the existing Metrics tab and the Phase 1 log-tail parser, then trace the vfolder file API used to read session files. Add tests for structured metrics and structured-over-parsed precedence, and update the user manual with the convention and the three requested integration snippets. Done means metrics.jsonl produces charts independently of stdout and the documented tests pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- typescript
- Domain
- documentation, frontend, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100