codefuse-ai / codefuse-ai/codefuse-chatbot

Can not make access with local llm

Open
#51 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.3k
Forks
147
PR merge metrics
No merged PRs in 30d

Description

### Checked other resources

- [X] I searched the Codefuse documentation with the integrated search.
- [X] I used the GitHub search to find a similar question and didn't find it.
- [X] I am sure that this is a bug in Codefuse-Repos rather than my code.
- [X] I added a very descriptive title to this issue.

### System Info

Windows

### Code Version

Latest Release

### Description

i already changed VLLM_MODEL_DICT in model_config.py and downloaded chatglm-6b in llm_models folder. Docker image has some dependencies error, so i used local services. Even though i modified the code as wrote in fastchat.md but i can not connect with local llm, also do not record a log at llm_api.log and sdfile_api.log .

### Example Code

#thats how i changed VLLM_MODEL_DICT
VLLM_MODEL_DICT = VLLM_MODEL_DICT or {
'chatglm2-6b': "chatglm-6b",
}

### Error Message and Stack Trace (if applicable)

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.