feyninc / feyninc/catsu

Support for OpenAI-compatible embedding endpoints

Abierto
#24 0 comentarios 2 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Rust
Estrellas
83
Forks
4
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

Several self-hosted embedding servers expose OpenAI-compatible `/embeddings` endpoints:
- [Infinity](https://github.com/michaelfeil/infinity)
- [vLLM](https://github.com/vllm-project/vllm)
- [Ollama](https://ollama.ai/)

The current OpenAI provider has a hardcoded `API_BASE_URL`. A configurable `base_url`
would make all these work without needing separate provider implementations.

**Possible API**
```python
client = Client()
response = client.embed(
model="BAAI/bge-small-en-v1.5",
inputs=["Hello world"],
base_url="http://localhost:7997" # or via env var
)
```

Since the request format is already OpenAI-compatible, only the URL routing would change.

Thanks for catsu, the unified API and model metadata catalog are really useful!

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.