epam / epam/statgpt-backend

refactor: simplify langchain/llm configs

Open
#150 1 comment 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
26
Forks
1
Avg merge
1d 17h
Merged PRs (30d)
25

Description

### StatGPT Backend version

latest

### What is the problem this feature will solve?

there is a separation of concerns issue between langchain settings and llm/embedding models settings:
- langchain settings allow to set default llm/embedding params via envvars (e.g. temperature)
- this does not introduce much value (we likely don't have a need to control such params via envvars, since we've moved to comprehensive llm models configs)
- however, it makes settings complex and unclear. it's much clearer to control these params using llm/embedding models configs

### What is the proposed feature or solution?

- remove following LangChainSettings fields:
- `default_model` (? could be a large refactoring and might be not worth it. but I think it's a good idea to explicitly specify the LLM model to use)
- `embedding_default_model` (??? even larger refactoring needed - probably keep it as is)
- `default_temperature`
- `default_seed`
- remove corresponding envvars, upd envvars markdown listing
- set `api_version` in langchain config, remove it from llm/embeddings model config

after removing envvars we'll need to update

- envvars readme files
- helms
- `.env.template`

### What alternatives have you considered?

keep as is

Contributor guide

Open the contributing guide

Research direction

Locate LangChainSettings and trace the listed default_model, embedding_default_model, default_temperature, default_seed, and api_version fields to their environment variables and model configuration. Review the envvars README files, Helm files, and .env.template before changing the configuration boundaries. Done means the agreed fields and variables are removed or retained consistently and all referenced configuration documentation and deployment templates are updated.

Written by the indexing model from the issue text.

Assessment

Tech stack
helm, python
Domain
ai, backend, devops
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.