bazel-contrib / bazel-contrib/rules_foreign_cc

Slow `configure_make` builds on windows

Open
#720 4 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Starlark
Stars
737
Forks
270
PR merge metrics
No merged PRs in 30d

Description

When running [configure_make](https://bazelbuild.github.io/rules_foreign_cc/0.4.0/configure_make.html) I noticed the builds were unbelievably slow in comparison to linux. This seems to be caused by the [symlink_to_dir](https://github.com/bazelbuild/rules_foreign_cc/blob/821d1efd24f4605fc96dd7f3f8db31c44d017368/foreign_cc/private/framework/toolchains/windows_commands.bzl#L121-L159) function or some other call that walks the directory tree:
```
Directory: C:\tmp\6ru5hwhq\execroot\rules_foreign_cc_examples\bazel-out\x64_windows-opt-exec-2B5CBBC6\bin\external\rules_foreign_cc\toolchains\make.build_tmpdir

Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 7/12/2021 2:15 PM build-aux
d----- 7/12/2021 2:15 PM doc
d----- 7/12/2021 2:15 PM lib
d----- 7/12/2021 2:15 PM m4
d----- 7/12/2021 2:15 PM mk
d----- 7/12/2021 2:15 PM po
d----- 7/12/2021 2:15 PM src
d----- 7/12/2021 2:15 PM tests
d----- 7/13/2021 6:57 AM WinRel
-a---- 7/13/2021 6:57 AM 93787 ABOUT-NLS
-a---- 7/13/2021 6:57 AM 54579 aclocal.m4
-a---- 7/13/2021 6:57 AM 3044 AUTHORS
-ar--- 7/12/2021 10:47 AM 5186 Basic.mk
-a---- 7/13/2021 6:57 AM 108 BUILD.bazel
-a---- 7/13/2021 6:57 AM 1216 build.cfg.in
-a---- 7/13/2021 6:57 AM 4531 build.sh
-a---- 7/13/2021 6:57 AM 5299 builddos.bat
-a---- 7/13/2021 6:57 AM 11011 build_w32.bat
-a---- 7/13/2021 6:57 AM 109692 ChangeLog
-a---- 7/13/2021 6:57 AM 534917 configure
-a---- 7/13/2021 6:57 AM 18566 configure.ac
-a---- 7/13/2021 6:57 AM 35147 COPYING
-a---- 7/12/2021 10:47 AM 5186 Makefile
-a---- 7/13/2021 6:57 AM 5315 makefile.com
-a---- 7/13/2021 6:57 AM 76437 Makefile.in
-a---- 7/13/2021 6:57 AM 72649 NEWS
-ar--- 7/12/2021 10:47 AM 7251 README
-a---- 7/13/2021 6:57 AM 2853 README.Amiga
-a---- 7/13/2021 6:57 AM 4495 README.customs
-ar--- 7/12/2021 10:47 AM 13737 README.DOS
-ar--- 7/12/2021 10:47 AM 6653 README.OS2
-a---- 7/13/2021 6:57 AM 21211 README.VMS
-ar--- 7/12/2021 10:47 AM 15479 README.W32
-a---- 7/13/2021 6:57 AM 200 SCOPTIONS
-a---- 7/13/2021 6:57 AM 937 vms_export_symbol_test.com
-a---- 7/13/2021 6:57 AM 3457543 WinRel.sbr
-a---- 7/13/2021 6:57 AM 32 WORKSPACE
```

And this takes quite a while to churn through (this is building [@rules_foreign_cc_examples//configure_with_bazel_transitive](https://github.com/bazelbuild/rules_foreign_cc/tree/821d1efd24f4605fc96dd7f3f8db31c44d017368/examples/configure_with_bazel_transitive))

Contributor guide

Open the contributing guide

Research direction

Start with foreign_cc/private/framework/toolchains/windows_commands.bzl, especially symlink_to_dir, and reproduce the slowdown with examples/configure_with_bazel_transitive on Windows. Trace which directory-walking operation dominates compared with Linux. Done means configure_make builds no longer spend excessive time traversing the build tree.

Written by the indexing model from the issue text.

Assessment

Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.