forrtproject / forrtproject/forrtproject.github.io

JUST-OS ideas

Open
#787 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
42
Forks
33
Avg merge
2d 6h
Merged PRs (30d)
19

Description

- Make top-k adaptive: over-retrieve, rerank, then apply a relative cutoff (keep within X% of the top score) bounded by a min and max k. - so that targeted questions don't get junk in, and broad questions get sufficient coverage
- Benchmark against a more modern 8B model, given that we don't use the full OpenScholar pipeline (e.g. Gemma 4 E4B, which scores 'highest' on AA Omniscience - i.e. low hallucinations)
- Benchmark against the actual OpenScholar pipeline (generation-feedback-regeneration)
- Curate our own benchmark, with expert-written OS answers (not sure how much of that exists already)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.