Clouditera / Clouditera/SecGPT

只有12g显存, 能否使用mini使用内存进行加速。

Open
#57 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3.1k
Forks
370
PR merge metrics
No merged PRs in 30d

Description

mini都是使用cpu进行推理的, 是否可以使用gpt进行加速。 比如我用的3060-12G . 是否能够充分发挥下显存。加快mini的推理速度。

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the mini inference entry point and checking how it selects CPU execution. Then determine whether the current runtime and model support the requested GPU path on a 12 GB RTX 3060, and define completion as a reproducible GPU-accelerated inference path with measured results. No files or tests are named in the issue, so repository research and design decisions are required.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.