huggingface / huggingface/chat-ui
Difficulty configuring multiple instances of the same model with distinct parameters
- Dominant language
- TypeScript
- Stars
- 11k
- Forks
- 1.7k
- Avg merge
- 21h 24m
- Merged PRs (30d)
- 56
Description
I am currently self-deploying an application that requires setting up multiple instances of the same model, each configured with different parameters. For example:
```
MODELS=`[{
"name": "gpt-4-0125-preview",
"displayName": "GPT 4",
"endpoints" : [{
"type": "openai"
}]
},
{
"name": "gpt-4-0125-preview",
"displayName": "GPT 4 temp 0",
"parameters": {
"temperature": 0.0
},
"endpoints" : [{
"type": "openai"
}]
}
]`
```
This results in a state which looks like that both models are active simultaneously.

However, in practice, I cannot activate the second model ("GPT 4 temp 0"); only "GPT 4" is utilized during chat operations. It appears as if the system defaults to the first model instance and ignores subsequent ones with the same model name.
I tried to distinguish between the models by modifying the `name` field and introducing an `id` field, using the appropriate model identifier. However, this approach resulted in a loss of model reference, indicating that these fields cannot be arbitrarily configured on the client side.
Is there a recommended approach to deploying two instances of the same model with varying parameters? Any guidance or suggestions on how to achieve this would be greatly appreciated.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.