github / github/codeql-cli-binaries

On MacOS, libtrace.dylib can't load into platform binaries for build instrumentation during DB creation

Aperta
#26 8 commenti 0 reazioni 0 assegnatari Vedi su GitHub
CLI
Lingua principale
Nessun dato sulla lingua
Stelle
1k
Fork
184
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

# Description
During database creation for compiled languages, it looks like CodeQL tries to add some instrumentation to your compiler to get more info. This fails for "platform binaries" on OSX, which are protected from `DYLD_INSERT_LIBRARIES` as well as`task_for_pid` and friends. For example, below I'm attempting to create a database for a particular revision of WebKit:

```
╭─user@box ~/project/webkit_stuff/WebKit.git
╰─$ uname -a
Darwin box.local 19.4.0 Darwin Kernel Version 19.4.0: Wed Mar 4 22:28:40 PST 2020; root:xnu-6153.101.6~15/RELEASE_X86_64 x86_64
╭─user@box ~/project/webkit_stuff/WebKit.git
╰─$ sw_vers
ProductName: Mac OS X
ProductVersion: 10.15.4
BuildVersion: 19E287
╭─user@box ~/project/webkit_stuff/WebKit.git
╰─$ codeql database create ~/project/codeql-home/webkit_regex_codeql --language=cpp --command="./Tools/Scripts/build-webkit --jsc-only --debug"
Initializing database at /Users/user/project/codeql-home/webkit_regex_codeql.
Running command [./Tools/Scripts/build-webkit, --jsc-only, --debug] in /Users/user/project/webkit_stuff/WebKit.git.
[2020-04-19 09:55:49] [build-err] error: /Users/user/Workspace/Xcode_10.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/install_name_tool: changing install names or rpaths can't be redone for: /Users/user/project/codeql-home/webkit_regex_codeql/working/copy-root/000001F5/Users/user/Workspace/Xcode_10.1.app/Contents/Developer/usr/bin/xcodebuild.semmle.00001694 (for architecture x86_64) because larger updated load commands do not fit (the program must be relinked, and you may need to use -headerpad or -headerpad_max_install_names)
[2020-04-19 09:55:49] [build-err] dyld: warning: could not load inserted library '/Users/user/project/codeql-home/codeql/tools/osx64/libtrace.dylib' into hardened process because no suitable image found. Did find:
[2020-04-19 09:55:49] [build-err] /Users/user/project/codeql-home/codeql/tools/osx64/libtrace.dylib: code signature in (/Users/user/project/codeql-home/codeql/tools/osx64/libtrace.dylib) not valid for use in process using Library Validation: mapping process is a platform binary, but mapped file is not
[2020-04-19 09:55:49] [build-err] /Users/user/project/codeql-home/codeql/tools/osx64/libtrace.dylib: stat() failed with errno=1
[2020-04-19 09:55:57] [build] + cmake -DPORT="JSCOnly" -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DCMAKE_BUILD_TYPE=Debug -DSHOW_BINDINGS_GENERATION_PROGRESS=1 -DDEVELOPER_MODE=ON -DENABLE_EXPERIMENTAL_FEATURES=ON "/Users/user/project/webkit_stuff/WebKit.git"
[2020-04-19 09:55:57] [build] -- The C compiler identification is AppleClang 10.0.0.10001145
[2020-04-19 09:55:57] [build] -- The CXX compiler identification is AppleClang 10.0.0.10001145
[2020-04-19 09:55:57] [build] -- Check for working C compiler: /Users/user/Workspace/Xcode_10.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
[2020-04-19 09:55:58] [build] -- Check for working C compiler: /Users/user/Workspace/Xcode_10.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc - works

[ SNIP - Compilation continues ]

[2020-04-19 10:07:24] [build] [100%] Linking CXX executable ../../../bin/jsc
[2020-04-19 10:07:24] [build] [100%] Built target jsc
[2020-04-19 10:09:21] [build] [100%] Linking CXX executable ../../../bin/testb3
[2020-04-19 10:09:22] [build] [100%] Built target testb3
[2020-04-19 10:09:22] [build] ====================================================================
[2020-04-19 10:09:22] [build] WebKit is now built (13m:33s).
[2020-04-19 10:09:22] [build] ====================================================================
Finalizing database at /Users/user/project/codeql-home/webkit_regex_codeql.
Successfully created database at /Users/user/project/codeql-home/webkit_regex_codeql.

```

You can see above the lines about `libtrace.dylib` injection failing, specifically:
```
[2020-04-19 09:55:49] [build-err] dyld: warning: could not load inserted library '/Users/user/project/codeql-home/codeql/tools/osx64/libtrace.dylib' into hardened process because no suitable image found. Did find:
[2020-04-19 09:55:49] [build-err] /Users/user/project/codeql-home/codeql/tools/osx64/libtrace.dylib: code signature in (/Users/user/project/codeql-home/codeql/tools/osx64/libtrace.dylib) not valid for use in process using Library Validation: mapping process is a platform binary, but mapped file is not
[2020-04-19 09:55:49] [build-err] /Users/user/project/codeql-home/codeql/tools/osx64/libtrace.dylib: stat() failed with errno=1
```
Disabling SIP doesn't appear to work around this, the outcome is the same. This is one example that under the hood probably uses things like `xcodebuild` and `clang`.

Database creation "finishes", but it's unclear how much information is actually missing. I actually don't know what all CodeQL tries to gather from the compilation process. How messed up will my database be if instrumentation fails during the build step?

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Start by reproducing the macOS build with `codeql database create` and the WebKit `build-webkit` command, then inspect the reported `tools/osx64/libtrace.dylib` injection failure for hardened platform binaries. Determine whether the completed database lacks required compilation information and document or correct the affected database-creation behavior.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
cpp, macos
Ambito
build-system, compilers, devtools
Tipo di issue
Bug
Difficoltà
4/5
Tempo stimato
3-5 giorni
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.