OpenBMB / OpenBMB/MiniCPM-o-Demo
量化问题
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 387
- Forks
- 81
- Avg merge
- 1h 59m
- Merged PRs (30d)
- 3
Description
你好，我这边已经成功部署运行，效果很好。
现在有2个方面的问题我想请问一下：
- 模型是否可以使用量化后的模型
- 目前代码是单用户对应独占worker的形式，如果当前使用好一点的cuda算力，是否可以做到单张显卡启动，多用户调用的形式。基于这些，是否可以用后台使用vllm部署加速，用工程代码的形式进行用户间的信息隔离，以达到单worker推理服务，实现多用户的并发效果？
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue names no files, tests, or entry points. First clarify whether the goal is quantized-model support, shared-worker multi-user inference, or both, and establish the expected isolation and concurrency behavior. Done should include agreed requirements and validation for the selected deployment approach.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100