codefuse-ai / codefuse-ai/codefuse-chatbot
Can not make access with local llm
- Dominant language
- Python
- Stars
- 1.3k
- Forks
- 147
- PR merge metrics
- No merged PRs in 30d
Description
### Checked other resources
- [X] I searched the Codefuse documentation with the integrated search.
- [X] I used the GitHub search to find a similar question and didn't find it.
- [X] I am sure that this is a bug in Codefuse-Repos rather than my code.
- [X] I added a very descriptive title to this issue.
### System Info
Windows
### Code Version
Latest Release
### Description
i already changed VLLM_MODEL_DICT in model_config.py and downloaded chatglm-6b in llm_models folder. Docker image has some dependencies error, so i used local services. Even though i modified the code as wrote in fastchat.md but i can not connect with local llm, also do not record a log at llm_api.log and sdfile_api.log .
### Example Code
#thats how i changed VLLM_MODEL_DICT
VLLM_MODEL_DICT = VLLM_MODEL_DICT or {
'chatglm2-6b': "chatglm-6b",
}
### Error Message and Stack Trace (if applicable)
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.