Questions about the Rouge metrics
Nobody has claimed this yet.
Assessment
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Newbie friendliness
- 25/100
- Issue type
- Bug
- Clarity
- Needs clarification
- Activity status
- Stale
- Tech stack
- jupyter-notebook
- Domain
- machine-learning
Research direction
No file, test, or entry point is named. Start by locating the Rouge evaluation and comparing the KB generation, standard, and ICL modes against the reported figures; done means identifying and documenting the cause of the metric differences.
Written by the indexing model from the issue text.
Description
1.Why is the kb's rouge metrics particularly low in generation mode, far from icl performance?
2.Why is the rouge metrics higher in standard mode when using KB than in generation mode? there is no essential difference between these two modes of using kb to generate answers.
- Dominant language
- Jupyter Notebook
- Stars
- 1.5k
- Forks
- 125
- Avg merge
- 22d 8m
- Merged PRs (30d)
- 1
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from microsoft/KBLaM
-
Difficulty 2/5 1-3 hours Newbie friendliness 62/100
-
Difficulty 4/5 3-5 days Newbie friendliness 35/100
-
Difficulty 4/5 3-5 days Newbie friendliness 25/100
-
Difficulty 5/5 Over a week Newbie friendliness 15/100
-
Difficulty 3/5 1-2 days Newbie friendliness 45/100
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
bancolombia/sentinel#21 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 85/100
infer-actively/pymdp#454 ·
-
bug
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
-
Difficulty 1/5 Under an hour Newbie friendliness 94/100
UKGovernmentBEIS/inspect_evals#2486 · 1 comment ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 88/100