bazelbuild / bazelbuild/bazel

Execlog parser should either prune or set a default for all the timing information.

Open
#24,071 0 comments 0 reactions 0 assignees View on GitHub
help wanted P3 team-Performance type: bug
Dominant language
Java
Stars
25.8k
Forks
4.6k
Avg merge
2d 18h
Merged PRs (30d)
75

Description

### Description of the bug:

Using something like:
```
> bazel-bin/src/tools/execlog/parser \
--log_path=/Users/fzakaria/Documents/exec_protoc1.log \
--log_path=/Users/fzakaria/Documents/exec_protoc2.log \
--output_path=/tmp/exec1.log.txt \
--output_path=/tmp/exec2.log.txt
```

I have included two compact exec logs to see how they are similar except for PATH and timings.
Using diff tools produces many differences on account of the timings.

```
> diff /tmp/exec1.log.txt /tmp/exec2.log.txt | head -n 30
67c67
< value: "/Users/fzakaria/Library/Caches/bazelisk/downloads/sha256/7dc88df4842a5b49830c55b174551445587ae60f027a67d0f847dbb35f1f40c6/bin:/Users/fzakaria/.pyenv/shims:/Users/fzakaria/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/opt/homebrew/bin:/Applications/iTerm.app/Contents/Resources/utilities:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/access-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/cloud-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/connect-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/k8saas-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/devprod-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/traffic-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/ksql-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/orchestrator-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/kafka-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/servicefoundations-beta:/Users/fzakaria/.zsh/plugins/powerlevel10k"
---
> value: "/Users/fzakaria/Library/Caches/bazelisk/downloads/sha256/7dc88df4842a5b49830c55b174551445587ae60f027a67d0f847dbb35f1f40c6/bin:/Users/fzakaria/.pyenv/shims:/Users/fzakaria/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/opt/homebrew/bin:/Applications/iTerm.app/Contents/Resources/utilities:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/access-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/cloud-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/connect-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/k8saas-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/devprod-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/traffic-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/ksql-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/orchestrator-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/kafka-ga:/Users/fzakaria/code/github.com/confluentinc/cc-dotfiles/bin/servicefoundations-beta:/Users/fzakaria/.zsh/plugins/powerlevel10k:/bin2/"
568,569c568
< runner: "disk cache hit"
< cache_hit: true
---
> runner: "darwin-sandbox"
573c572
< hash: "630a6822f70ec0fbb920b1ecfe78137657d6a46d989df99034b4f163913c7684"
---
> hash: "c36a3b77ddd78a42d736c42c8c2ebb341548049db36a3e6591d440d8a004fc6b"
579c578
< nanos: 8000000
---
> nanos: 350000000
581,582c580,581
< fetch_time {
< nanos: 2000000
---
> execution_wall_time {
> nanos: 350000000
584,585d582
< input_bytes: 1812815
< input_files: 59
587,588c584,585
< seconds: 1729717622
< nanos: 557000000
---
```

[exec_protoc2.log](https://github.com/user-attachments/files/17498813/exec_protoc2.log)
[exec_protoc1.log](https://github.com/user-attachments/files/17498812/exec_protoc1.log)

My vote would be for this to be the default, but I could imagine it needing to be behind a flag to retain original timings for whatever other workflows exist.

Contributor guide

Open the contributing guide

Research direction

Start with the src/tools/execlog/parser entry point and run it against the two attached exec logs. Compare the generated outputs, focusing on the timing fields and the reported differences. Done means timing information is consistently pruned or given defaults so otherwise similar logs produce comparable output.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
build-system, tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.