conductor-oss / conductor-oss/python-sdk
Support for asyncio based workers
- 主要语言
- Python
- 星标
- 104
- 派生
- 42
- 平均合并
- 2 天 1 小时
- 30 天内合并 PR
- 3
描述
The current worker implementation model uses multiprocessing for parallel execution but each process blocks on task execution (though it might be waiting on I/O). Is there a possibility to use asyncio/threaded approach to alleviate this problem and think of a model where processes could be used for parallelization (using multiple cores) and each process has an event loop/ThreadPool to poll and execute tasks to maximize performance on each core ?
I'd be happy to discuss more on this and submit a PR if required as well. Thanks for all the awesome work so far 👍
贡献指南
这个仓库没有索引到贡献指南
调研方向
首先检查当前的 multiprocessing worker 实现及其阻塞式任务执行路径。在开始编码之前,明确所需的 asyncio/threaded 进程与事件循环模型、性能目标和验收标准;该 issue 目前描述的是设计讨论,而不是范围明确的变更。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- python
- 领域
- backend, distributed-systems, performance
- Issue 类型
- 功能
- 难度
- 5/5
- 预计耗时
- 一周以上
- 活跃度
- 停滞
- 描述清晰度
- 需要澄清
- 新手友好度
- 28/100