microsoft / microsoft/azure-devops-python-api
Finding a specific build log is incredibly slow
还没有人认领这个 Issue。
- 主要语言
- Python
- 星标
- 684
- 派生
- 218
- 平均合并
- 8 天 10 小时
- 30 天内合并 PR
- 1
描述
I have a script where I am trying to retrieve build logs for all PRs we have that are failing so I can parse the logs and collate the data.
I can get all of the PRs that are failing a particular check with the Github API pretty easily. I end up with detail links which all point to our internal ADO project.
It when I need to pull the logs from ADO that things get really slow. The code basically looks like this:
# PHASE 2: Collect logs for failed LintDiff runs from Azure DevOps
failure_logs = []
ado = get_ado_client()
for run in failed_lint_diff_runs:
link = run.details_url
build_id = get_build_id(link)
build_client = ado.clients_v7_1.get_build_client()
build_logs = build_client.get_build_logs("internal", build_id)
for log in build_logs:
lines = build_client.get_build_log_lines("internal", build_id, log.id)
if "Starting: LintDiff" in lines[0]:
print(f"Found LintDiff log for build {build_id} at log {log.id}")
failure_logs.append(lines)
break
When I run this, it takes about 20 minutes to run... all of the time is spent repeatedly calling get_build_log_lines to examine just the first line of the build log to see if it is the one I care about. The output looks like:
PRs with LintDiff failures: 16
Found LintDiff log for build 3497966 at log 297
Found LintDiff log for build 3493487 at log 375
Found LintDiff log for build 3497244 at log 242
Found LintDiff log for build 3471732 at log 295
Found LintDiff log for build 3468386 at log 417
Found LintDiff log for build 3468168 at log 395
Found LintDiff log for build 3459808 at log 307
Found LintDiff log for build 3475219 at log 296
Found LintDiff log for build 3469709 at log 307
Found LintDiff log for build 3497648 at log 310
Found LintDiff log for build 3462035 at log 212
Found LintDiff log for build 3370030 at log 342
Found LintDiff log for build 3349947 at log 391
Found LintDiff log for build 3328138 at log 306
Found LintDiff log for build 3327852 at log 304
Found LintDiff log for build 3327331 at log 306
You can see, we have a lot of build logs, so much of the time is wasted. I couldn't find anything about the build log object that is identifiable. The ID changes all the time, and while in the UI it has a consistent name, that name doesn't seem to be included here. If it were, it would save a ton of time.
Is there a better way to collect this information?
贡献指南
这个仓库没有索引到贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
从 issue 中展示的 Azure DevOps Python 客户端调用开始:get_build_client()、get_build_logs() 和 get_build_log_lines()。检查相应的 build-log API 行为以及可用于在下载日志行之前识别日志的元数据。完成的标准是:无需反复获取每个日志的内容,即可找到目标 LintDiff 日志。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- azure, python
- 领域
- api, devops
- Issue 类型
- 功能
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100