ByteDance-Seed / ByteDance-Seed/Bagel
如何进行推理加速？
Open
- Dominant language
- Python
- Stars
- 6.2k
- Forks
- 545
- PR merge metrics
- No merged PRs in 30d
Description
InterleaveInferencer 似乎只支持串行调用，不支持并发 or 并批。
想问下是否有什么推理加速的方法？比如支持用哪些框架进行部署？
是否可以提供一些示例？感谢❤
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the InterleaveInferencer entry point and inspect how inference is currently called serially. Clarify the desired concurrency or batching behavior and which deployment frameworks and examples are expected; no specific files or tests are named in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100