FastChat Code base help
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
Hello lm-sys team, I was looking through the code base of Fastchat for Chatbot arena code but I Couldn't find it.
I am specifically interested on the model selection when user asks a question. I have gone through your Research Paper to Understand your Elo Rankings and Active sampling of Model Selection.
It want to explore the LLM Selection using active sampling and how you are doing it.
It would be helpful if you can give more explanation or insights on it.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file, test, or entry point is named. Start by locating the Chatbot Arena model-selection code in the FastChat codebase and compare it with the cited Elo Rankings and Active Sampling research paper. Done would require a concrete explanation or documentation of how active sampling selects models, but the requested scope is not defined.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100