EleutherAI / EleutherAI/lm-evaluation-harness
Add HuggingFace Text-Generation-Interface Support
Open
- Dominant language
- Python
- Stars
- 14k
- Forks
- 3.6k
- Avg merge
- 4d 2h
- Merged PRs (30d)
- 35
Description
Add HuggingFace Text-Generation-Interface Support: https://github.com/huggingface/text-generation-inference
Contributor guide
Research direction
No files, tests, or entry points are named. Start by reading the linked Hugging Face Text Generation Inference project and the harness's existing model integrations to determine the required support boundary. Done should mean the integration is implemented and its evaluation behavior is covered by appropriate tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface, python
- Domain
- backend, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100