capreolus-ir / capreolus-ir/diffir
Add summary of metrics to landing page
- Dominant language
- Python
- Stars
- 20
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
Based on earlier discussions, it would be nice to display the following on the landing page to help assess a single run:
1. Number of unjudged docs (in top K)
2. Metrics with upper and lower bounds (based on assumptions about unjudged docs)
And to assess pairs of runs:
3. Wins/losses/draws between the two runs (e.g., number of queries with a > 0.01 difference in a metric)
Contributor guide
No contributing guide indexed for this repository
Research direction
No implementation files or tests are identified in the issue. Start by locating the landing-page entry point and the code that computes or presents run metrics, then determine how single-run bounds and pairwise wins, losses, and draws should be represented. Done means the landing page displays all three requested summaries for one run and for a pair of runs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- analytics, frontend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100