google / google/oss-fuzz

`build_fuzzers --sanitizer=... project local-dir` can't be run a few times in a row

Open
#9,249 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Shell
Stars
12.6k
Forks
2.9k
Avg merge
2d 2h
Merged PRs (30d)
62

Description

FI uses `./infra/helper.py build_fuzzers` to build projects with three different sanitizers and that works only when build scripts clean up build directories themselves by running `make clean`, `ninja -C ./build/ clean`, `git clean -dxff` or something like that. To avoid editing all the build scripts I think it would be better if `./helper.py` could destroy any changes to build directories itself. One option would be to set up overlays by analogy with how the ":O" option in podman works
> The :O flag tells Podman to mount the directory from the host as a temporary storage using the overlay file system. The container processes can modify content within the mountpoint which is stored in the container storage in a separate directory. In overlay terms, the source directory will be the lower, and the container storage directory will be the upper. Modifications to the mount point are destroyed when the container finishes executing, similar to a tmpfs mount point being unmounted.

(`docker` doesn't support it out of the box as far as I know)

Here's what happens when `build_fuzzers` is run two times in a row with different sanitizers when build scripts don't run `git clean -dxff` to clean up build directories:
```sh
$ ./infra/helper.py build_fuzzers --clean --sanitizer=address dbus-broker ~/dbus-broker
...
$ ./infra/helper.py build_fuzzers --clean --sanitizer=coverage dbus-broker ~/dbus-broker
...
+ meson -Db_lundef=false -Dlauncher=false build^M
Directory already configured.^M
^M
...
/usr/bin/ld: /usr/bin/ld: DWARF error: invalid or unhandled FORM value: 0x25^M
build/src/libbus-static.a.p/dbus_message.c.o: in function `message_new_incoming':^M
message.c:(.text.message_new_incoming[message_new_incoming]+0x29): undefined reference to `__asan_option_detect_stack_use_after_return'^M
...
/usr/bin/ld: build/subprojects/libcutf8-1/src/libcutf8-1.so.0.p/c-utf8.c.o: in function `asan.module_ctor':^M
c-utf8.c:(.text.asan.module_ctor+0x5): undefined reference to `__asan_init'^M
/usr/bin/ld: c-utf8.c:(.text.asan.module_ctor+0xa): undefined reference to `__asan_version_mismatch_check_v8'^M
clang-15: ^[[0;1;31merror: ^[[0m^[[1mlinker command failed with exit code 1 (use -v to see invocation)^[[0m^M
ERROR:root:Building fuzzers failed.
```

I "fixed" it locally by pointing `docker` to `podman` and applying the following patch
```diff
diff --git a/infra/helper.py b/infra/helper.py
index 8d17cb8c..caef6825 100755
--- a/infra/helper.py
+++ b/infra/helper.py
@@ -701,7 +701,7 @@ def build_fuzzers_impl( # pylint: disable=too-many-arguments,too-many-locals,to

command += [
'-v',
- '%s:%s' % (_get_absolute_path(source_path), workdir),
+ '%s:%s:O' % (_get_absolute_path(source_path), workdir),
]

command += [
```

It came up in https://github.com/ossf/fuzz-introspector/pull/637 and https://github.com/google/oss-fuzz/pull/9243.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.