michaelfeil / michaelfeil/infinity
Support Integration with KServe
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 206
- PR merge metrics
- No merged PRs in 30d
Description
Feature request
Kserve is a Kubernetes based engine for predictive and generative AI models and provides abstraction for popular model servers like Huggingface TEI (https://github.com/kserve/kserve/pull/3743), Tensorflow,PyTorch etc. Request to support Infinity as a model serving engine in Kserve too.
Motivation
Many organizations using OSS are using Kserve for predictive model deployment, and are attempting to use the same for embedding and generative model deployment. Having Infinity as a model serving engine would help to avoid a separate deployment for infinity altogether Ref :#314 .
Your contribution
NA
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No repository file or test is identified. Start by reviewing issue #314 and KServe's model-serving integration, including the referenced KServe pull request, to determine the integration boundary. Done means Infinity is supported as a KServe model-serving engine without requiring a separate deployment.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, python
- Domain
- ai, infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100