huggingface / huggingface/chat-ui

Difficulty configuring multiple instances of the same model with distinct parameters

Open
#875 1 comment 1 reaction 0 assignees View on GitHub
Dominant language
TypeScript
Stars
11k
Forks
1.7k
Avg merge
21h 24m
Merged PRs (30d)
56

Description

I am currently self-deploying an application that requires setting up multiple instances of the same model, each configured with different parameters. For example:

```
MODELS=`[{
"name": "gpt-4-0125-preview",
"displayName": "GPT 4",
"endpoints" : [{
"type": "openai"
}]
},
{
"name": "gpt-4-0125-preview",
"displayName": "GPT 4 temp 0",
"parameters": {
"temperature": 0.0
},
"endpoints" : [{
"type": "openai"
}]
}
]`
```

This results in a state which looks like that both models are active simultaneously.
![image](https://github.com/huggingface/chat-ui/assets/99467346/0ed9d506-d413-45b5-b959-92e872875748)

However, in practice, I cannot activate the second model ("GPT 4 temp 0"); only "GPT 4" is utilized during chat operations. It appears as if the system defaults to the first model instance and ignores subsequent ones with the same model name.

I tried to distinguish between the models by modifying the `name` field and introducing an `id` field, using the appropriate model identifier. However, this approach resulted in a loss of model reference, indicating that these fields cannot be arbitrarily configured on the client side.

Is there a recommended approach to deploying two instances of the same model with varying parameters? Any guidance or suggestions on how to achieve this would be greatly appreciated.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.