channable / channable/opsqueue
CI keeps failing due to timeout
- 主要语言
- Rust
- 星标
- 96
- 派生
- 2
- 平均合并
- 1 小时 2 分钟
- 30 天内合并 PR
- 2
描述
When i open a PR, i often have to run the job a couple of times before it completes in time.
We've had issues before with our integration tests being flaky and becoming deadlocked indefinitely. #6 introduced the 20 second timeout for integration tests, so they fail more quickly once they become deadlocked.
It's also difficult to see what exactly causes the failure. A first step towards resolving this issue could be to see what we can do to improve the output that we get when a test times out on CI, because at time of writing this is just a massive stack trace with mostly callsites originating from pytest plugins and the like.
Either the time-out is just too short for CI, or we're still getting deadlocks. So far, i've not been able to really reproduce any deadlocks by running the tests locally. We could try to relax that timeout a bit more, but not by too much. As the comment above the timeout configuration states, we're already being quite generous with our time limit.
**Examples**
The last four PRs have all seen at least one failure of the integration tests due to it exceeding the time limit:
- #19
- [failed push](https://channable.semaphoreci.com/workflows/56fef772-2c93-4009-a632-fa1b51fe5aa4)
- #18
- [failed push](https://channable.semaphoreci.com/jobs/ef466f0d-4f1c-4e1c-9e8f-883d47d118c5)
- [failed re-run](https://channable.semaphoreci.com/jobs/c496f290-f42b-46ea-9605-13204f6fcae5)
- #17
- [failed push](https://channable.semaphoreci.com/jobs/ed000520-408d-4222-a674-02833521bdf9)
- #16
- [failed merge by Hoff](https://github.com/channable/opsqueue/pull/16#issuecomment-3376018058)
贡献指南
这个仓库没有索引到贡献指南
调研方向
从 issue 中引用的集成测试超时配置开始,并检查与 PRs #16–#19 关联的 CI 作业。将超时输出与本地运行结果进行比较,以确定失败是由死锁导致,还是 CI 限制不足。完成的标准是已确定原因,并且超时失败会生成可采取行动的输出,或已进行有充分依据的超时调整。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- rust
- 领域
- ci-cd, testing
- Issue 类型
- 缺陷
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 活跃度
- 停滞
- 描述清晰度
- 需要澄清
- 新手友好度
- 35/100