如何对 'http://127.0.0.1:9009/v1/completions' 进行 测试

问题描述 / Issue Description

修改 api_url='http://127.0.0.1:9009/v1/completions' 后，执行测试报错 ，是不是不支持呢

使用的工具 / Tools Used

• Native / 原生框架

执行的代码或指令 / Code or Commands Executed

from evalscope import TaskConfig, run_task
from evalscope.constants import EvalType

task_cfg = TaskConfig(
model='DeepSeek-R1',
api_url='http://127.0.0.1:9009/v1/completions',
api_key='EMPTY',
eval_type=EvalType.SERVICE,
datasets=[
'mmlu',
],
dataset_args={
'mmlu': {
'dataset_id': '/home/zte/workspace/xjf/script/math_500_mmlu_evalscope/evalscopedata/mmlu',
"filters": {"remove_until": ""
#'output_types':['multiple_choice_logits']
}
},
eval_batch_size=16, # num of workers to seed requests
generation_config={
'max_tokens': 16000, # avoid exceed max length
#'do_sample': true,
#'temperature': 0.6,
#'top_p': 0.95,
#'top_k': 0.40,
#'n': 1 # num of repeat for each prompt (note lmdeploy only support n=1)
},
#limit=10,
stream=True
)

错误日志 / Error Log

Getting answers: 0%| | 0/728 [00:02<?, ?it/s]
Traceback (most recent call last):
File "/aih/AIH/pip/tianshu_math500_test_pip_install/llama-8b_evalscope_math_test_completion_nostream.py", line 30, in
run_task(task_cfg=task_cfg)
File "/usr/local/lib/python3.10/site-packages/evalscope/run.py", line 31, in run_task
return run_single_task(task_cfg, run_time)
File "/usr/local/lib/python3.10/site-packages/evalscope/run.py", line 44, in run_single_task
return evaluate_model(task_cfg, outputs)
File "/usr/local/lib/python3.10/site-packages/evalscope/run.py", line 117, in evaluate_model
res_dict = evaluator.eval()
File "/usr/local/lib/python3.10/site-packages/evalscope/collections/evaluator.py", line 265, in eval
answers = self.get_answers()
File "/usr/local/lib/python3.10/site-packages/evalscope/collections/evaluator.py", line 212, in get_answers
answer_list, samples = future.result()
File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 451, in result
return self.__get_result()
File "/usr/local/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/usr/local/lib/python3.10/site-packages/evalscope/collections/evaluator.py", line 37, in get_answer
answer_ds: List[dict] = self.model_adapter.predict(inputs=input_prompts, infer_cfg=infer_cfg)
File "/usr/local/lib/python3.10/site-packages/evalscope/models/server_adapter.py", line 55, in predict
response = self.process_single_input(input_item, infer_cfg)
File "/usr/local/lib/python3.10/site-packages/evalscope/models/server_adapter.py", line 72, in process_single_input
response = self.send_request(request_json)
File "/usr/local/lib/python3.10/site-packages/evalscope/models/server_adapter.py", line 111, in send_request
response = self.client.chat.completions.create(**request_json)
File "/usr/local/corex-4.2.0/lib64/python3/dist-packages/openai/_utils/_utils.py", line 279, in wrapper
return func(*args, **kwargs)
File "/usr/local/corex-4.2.0/lib64/python3/dist-packages/openai/resources/chat/completions.py", line 859, in create
return self._post(
File "/usr/local/corex-4.2.0/lib64/python3/dist-packages/openai/_base_client.py", line 1283, in post
return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
File "/usr/local/corex-4.2.0/lib64/python3/dist-packages/openai/_base_client.py", line 960, in request
return self._request(
File "/usr/local/corex-4.2.0/lib64/python3/dist-packages/openai/_base_client.py", line 1064, in _request
raise self._make_status_error_from_response(err.response) from None
openai.NotFoundError: Error code: 404 - {'detail': 'Not Found'}
/usr/local/lib/python3.10/tempfile.py:860: ResourceWarning: Implicitly cleaning up <TemporaryDirectory '/tmp/tmpsas8wzt3'>
_warnings.warn(warn_message, ResourceWarning)
root@49d005cc5a10:/aih/AIH/pip/tianshu_math500_test_pip_install# cd /usr/local/lib/pyth

运行环境 / Runtime Environment

• 操作系统 / Operating System:

  • Ubuntu

• Python版本 / Python Version:

  • 3.10

其他信息 / Additional Information

如果有其他相关信息，请在此处提供。 / If there is any other relevant information, please provide it here.