binary-husky / binary-husky/gpt_academic
[Bug]: 无法调用本地模型
- Dominant language
- Python
- Stars
- 71.3k
- Forks
- 8.3k
- PR merge metrics
- No merged PRs in 30d
Description
### Installation Method | 安装方法与平台
Pip Install (I used latest requirements.txt)
### Version | 版本
Latest | 最新版
### OS | 操作系统
Linux
### Describe the bug | 简述
调用本地Deepseek API出错:
```
11:11 | get_encoder :60 | 加载tokenizer完毕
Traceback (most recent call last):
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/gradio/routes.py", line 422, in run_predict
output = await app.get_blocks().process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/gradio/blocks.py", line 1323, in process_api
result = await self.call_function(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/gradio/blocks.py", line 1067, in call_function
prediction = await utils.async_iteration(iterator)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/gradio/utils.py", line 336, in async_iteration
return await iterator.__anext__()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/gradio/utils.py", line 329, in __anext__
return await anyio.to_thread.run_sync(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2461, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 962, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/gradio/utils.py", line 312, in run_sync_iterator_async
return next(iterator)
^^^^^^^^^^^^^^
File "/kaggle/working/gpt_academic/toolbox.py", line 136, in decorated
yield from f(txt_passon, llm_kwargs, plugin_kwargs, chatbot_with_cookie, history, system_prompt, *args)
File "/kaggle/working/gpt_academic/request_llms/bridge_all.py", line 1399, in predict
method = model_info[llm_kwargs['llm_model']]["fn_with_ui"] # 如果这里报错,检查config中的AVAIL_LLM_MODELS选项
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'default'
````
我很确认 `default` 模型写在AVAIL_LLM_MODELS里,而且API请求地址也修改为自己的地址了。但是还是调用不了,折腾了好几个小时。实在无助所以求解决!
### Screen Shot | 有帮助的截图

### Terminal Traceback & Material to Help Reproduce Bugs | 终端traceback(如有) + 帮助我们复现的测试材料样本(如有)
(gptac_venv) root@163a3a446619:/kaggle/working/gpt_academic# python main.py
11:11 | ..up_logging:60 | 所有对话记录将自动保存在本地目录gpt_log/admin/chat_secrets.log, 请注意自我隐私保护哦!
11:11 | .._lru_cache:95 | [PROXY] 网络代理状态:未配置。无代理状态下很可能无法访问OpenAI家族的模型。建议:检查USE_PROXY选项是否修改。
11:11 | check_proxy :35 | 代理配置 无, 代理所在地:Australia
11:11 | .._lru_cache:86 | [API_KEY] 本项目现已支持OpenAI和Azure的api-key。也支持同时填写多个api-key,如API_KEY="openai-key1,openai-key2,azure-key3"
11:11 | .._lru_cache:87 | [API_KEY] 您既可以在config.py中修改api-key(s),也可以在问题输入区输入临时的api-key(s),然后回车键提交后即可生效。
11:11 | .._lru_cache:91 | [API_KEY] 您的 API_KEY(***)不满足任何一种已知的密钥格式,请在config文件中修改API密钥之后再运行(详见`https://github.com/binary-husky/gpt_academic/wiki/api_key`)。
11:11 | ..ayed_tasks:337 | 如果浏览器没有自动打开,请复制并转到以下URL:
11:11 | ..ayed_tasks:338 | 「暗色主题已启用(支持动态切换主题)」: http://localhost:35909
11:11 | ..up_modules:225 | 正在执行一些模块的预热 ...
11:11 | get_encoder :58 | 正在加载tokenizer,如果是第一次运行,可能需要一点时间下载参数
11:11 | get_encoder :60 | 加载tokenizer完毕
11:11 | get_encoder :58 | 正在加载tokenizer,如果是第一次运行,可能需要一点时间下载参数
11:11 | get_encoder :60 | 加载tokenizer完毕
Traceback (most recent call last):
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/gradio/routes.py", line 422, in run_predict
output = await app.get_blocks().process_api(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/gradio/blocks.py", line 1323, in process_api
result = await self.call_function(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/gradio/blocks.py", line 1067, in call_function
prediction = await utils.async_iteration(iterator)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/gradio/utils.py", line 336, in async_iteration
return await iterator.__anext__()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/gradio/utils.py", line 329, in __anext__
return await anyio.to_thread.run_sync(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 2461, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/anyio/_backends/_asyncio.py", line 962, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/conda/envs/gptac_venv/lib/python3.11/site-packages/gradio/utils.py", line 312, in run_sync_iterator_async
return next(iterator)
^^^^^^^^^^^^^^
File "/kaggle/working/gpt_academic/toolbox.py", line 136, in decorated
yield from f(txt_passon, llm_kwargs, plugin_kwargs, chatbot_with_cookie, history, system_prompt, *args)
File "/kaggle/working/gpt_academic/request_llms/bridge_all.py", line 1399, in predict
method = model_info[llm_kwargs['llm_model']]["fn_with_ui"] # 如果这里报错,检查config中的AVAIL_LLM_MODELS选项
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: 'default'
我的API请求格式是这样子的,符合OpenAI请求格式:
```
import openai
def run():
client = openai.Client(base_url="http://8.155.8.136:47777/v1", api_key="abcd************xyz")
r = client.chat.completions.create(
model="default",
messages=[
{"role": "system", "content": """你将扮演AI聊天机器人与用户聊天"""},
{"role": "user", "content": """现在几点了?"""},
],
temperature=0.2,
max_tokens=20480,
stream=True,
timeout=9999
)
for chunk in r:
print(chunk.choices[0].delta.content, end='')
if __name__ == '__main__':
run()
```
SGlang部署的deepseek 671B。纯python代码可以调用,但是无法接入到gpt_academic
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in request_llms/bridge_all.py at the predict entry point shown in the traceback, then inspect the configuration containing AVAIL_LLM_MODELS and the selected llm_model value. Reproduce the local-model request with the reported default key. Done means the application can invoke the configured local DeepSeek-compatible API without the KeyError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, api, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100