Clouditera / Clouditera/SecGPT

为什么生成的耗时非常久,平均一个词要1-2秒,整个句子生成用了十几分钟。

Open
#45 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3.1k
Forks
370
PR merge metrics
No merged PRs in 30d

Description

你好,我使用的2张A100*40G,用的是你们14B的大模型执行webdemo.py,交互体验很差,问一个问题要生成答案好久,平均一个回复要十几分钟。请问你们测试也是这样情况吗?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading and running webdemo.py with the reported 14B model configuration on two A100 40G GPUs. Measure token and full-response latency, then compare the observed behavior with the expected experience described in the issue. Done means the cause of the slow generation is identified and a focused fix or clear performance diagnosis is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.