jellydn / jellydn/flowly

Add an interactive evaluation dashboard

Open
#154 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
3
Forks
0
Avg merge
15h 21m
Merged PRs (30d)
31

Description

## Summary

Add a read-only web dashboard for exploring Flowly evaluation runs and historical leaderboards. This is an optional follow-up to #38.

## Scope

- Browse persisted benchmark runs by suite and model
- Compare quality, latency, tokens, cost, tool success, patch applicability, and human acceptance
- Filter historical results and inspect scenario-level evidence
- Reuse the existing report store and metrics instead of creating a second evaluation data model

## Acceptance criteria

- [ ] The dashboard renders persisted reports and historical leaderboard data
- [ ] Model and run comparisons expose scenario-level metrics and regression status
- [ ] Empty, unknown-cost, and failed-run states are clear
- [ ] The UI is read-only and does not expose provider keys or other secrets
- [ ] Tests and user documentation cover the dashboard workflow

## Non-goals

- Running benchmarks from the dashboard
- Hosted/public report sharing (tracked separately)
- Automatic model routing recommendations (tracked separately)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the existing report store and evaluation metrics, then review the related follow-up issue #38 to understand the existing workflow. Define the dashboard around persisted reports and historical leaderboards, keeping it read-only. Done means the acceptance criteria are met, including scenario-level comparisons, clear empty and failed states, tests, and user documentation.

Written by the indexing model from the issue text.

Assessment

Tech stack
typescript
Domain
analytics, frontend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.