aipotheosis-labs / aipotheosis-labs/aci

Connect to Azure OpenAI and OpenAI API-Compatible Providers

Abierto
#512 1 comentario 0 reacciones 1 asignado Reclamado por @hanyixxx Ver en GitHub
Lenguaje dominante
Python
Estrellas
4.9k
Forks
484
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

## Required Pre-requisites

- [x] I have read the [Documentation](https://www.aci.dev/docs)
- [x] I have searched the [Issue Tracker](https://github.com/aipotheosis-labs/aci/issues) and [Discussions](https://github.com/aipotheosis-labs/aci/discussions) that this hasn't been reported yet.
- [x] Consider asking in [Discussions](https://github.com/aipotheosis-labs/aci/discussions) first

## Motivation

Be able to use different openai compatible providers or services.
Example using azure openai or a self hosted inference services (openai compatible) like lmstudio, ollama. Be able to change the embedding model if needed.

## Proposed Solution

### Proposed Solution:
The suggested solution will be through abstracting the creation of the openai client to a new singleton factory method in the common folder, e.g. "openai_client.py".

The OpenAI client support passing additional argument to control the non default base_url, azure endpoint, api version, project, organization.
- The singleton factory can utilize the additional OpenAI variables while creating the client and abstracted from the other python files that utilize the client.
- Abstract caching of multiple clients based on their API Keys, to support CLI vs Server clients (assuming we connect to the same inference server, but using different application keys )
- Add an extra environment to specify which client_type we are creating ( `openai` vs `azure` ) to create the client using OpenAI or AzureOpenAI.

### Advantages:
- Adding an easy configurable way for switching the OpenAI providers
- Developers with no OpenAI access can use local hosted services like Ollama / LMStudio / other providers for local development.

**Notes for Local hosted openai provider services and docker compose:**
- If you are consideration using non openai models, make sure you select a model that support 1024 to support the postgres/pgvector needs.
- If you are using ollama/Lmstudio/self hosted provider, you need to make it accessible to your backend server within the docker network.
- For Ollama if you use docker compose for development, you may need to run ollama after setting this environment variable on your machine (export OLLAMA_HOST="0.0.0.0:11434") to be able to access it form non localhost ip.
- For LMStudio: you will need to enable "serve on Local Network" to listen to non localhost ip -> 0.0.0.0:1234

I already made changes locally to test running the backend server with Azure OpenAI and connected to Ollama to use a different model ("mxbai-embed-large").

---
For using non-standard openai services embeddings with pgvector, you need to choose a compatible embedding model that support dimension = 1024

```env
# if using OpenAI, you can use these variables in `.env.local`
OPENAI_CLIENT_TYPE=openai
# OpenAI Org id
# OPENAI_ORG_ID=
# OpenAI Project id
# OPENAI_PROJECT_ID=
## Keys
SERVER_OPENAI_API_KEY=
CLI_OPENAI_API_KEY=

## if using Azure, you can use these variables in `.env.local`
OPENAI_CLIENT_TYPE=azure
AZURE_OPENAI_ENDPOINT=https://.openai.azure.com/
OPENAI_API_VERSION=2024-02-01
## Keys
SERVER_OPENAI_API_KEY=
CLI_OPENAI_API_KEY=

# if using OpenAI compatible api utilize the base url (e.g. OLLAMA) , you can use these variables in `.env.local`
OPENAI_CLIENT_TYPE=openai
OPENAI_BASE_URL=http://localhost:11434/v1
## Keys
SERVER_OPENAI_API_KEY=
CLI_OPENAI_API_KEY=

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.