Codeql database create fails when building mozilla
- 主要語言
- CodeQL
- 星號
- 10.1k
- 分支
- 2.1k
- 平均合併
- 2 天 15 小時
- 30 天內合併 PR
- 141
描述
I'm trying to build mozilla from [this repo](https://github.com/mozilla/gecko-dev) and create a cpp codeql database.
In order to that, I first run `./mach configure` and after it's done, I run the following command to create the database:
`~/codeql/codeql database create mozilla4 --language=c-cpp --command="./mach build"`
Which builds mozilla successfully, but it fails in the importing trap file stage. I have included the logs for [creating](https://drive.google.com/file/d/1Npi4LVb7fzHoALmQrlDPKofnpYheE8lg/view?usp=drive_link) the database and [importing](https://drive.google.com/file/d/15J_n7OoP4ifb9TTmXnkHBDpYpgVDI596/view?usp=sharing) dataset.
A glance of the errors is like:
```
[ERROR] 14135855_0.trap.br, 1: java.io.IOException: Brotli stream decoding failed
org.brotli.dec.BrotliInputStream.read(BrotliInputStream.java:167)
com.semmle.inmemory.trap.TrapInputStream.read(TrapInputStream.java:60)
com.semmle.inmemory.trap.TrapScanner.fill(TrapScanner.java:451)
com.semmle.inmemory.trap.TrapScanner.ensureNext(TrapScanner.java:428)
com.semmle.inmemory.trap.TrapScanner.nextToken(TrapScanner.java:61)
com.semmle.inmemory.trap.TRAPReader.scanTuplesAndLabels(TRAPReader.java:493)
com.semmle.inmemory.trap.TRAPLinker$TrapLinkDirectiveScanner.scanTuplesAndLabels(TRAPLinker.java:311)
com.semmle.inmemory.trap.TRAPReader.importTuples(TRAPReader.java:414)
com.semmle.inmemory.trap.TRAPReader.importTuples(TRAPReader.java:400)
com.semmle.inmemory.trap.TRAPLinker.lambda$getTasks$2(TRAPLinker.java:215)
com.semmle.util.concurrent.FutureUtils.lambda$mapAsync_$8(FutureUtils.java:161)
java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source)
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.base/java.lang.Thread.run(Unknown Source)
... caused by:
org.brotli.dec.BrotliRuntimeException: Corrupted Huffman code histogram
org.brotli.dec.Decode.readComplexHuffmanCode(Decode.java:591)
org.brotli.dec.Decode.readHuffmanCode(Decode.java:613)
org.brotli.dec.Decode.readMetablockPartition(Decode.java:783)
org.brotli.dec.Decode.readMetablockHuffmanCodesAndContextMaps(Decode.java:825)
org.brotli.dec.Decode.decompress(Decode.java:1110)
org.brotli.dec.BrotliInputStream.read(BrotliInputStream.java:162)
com.semmle.inmemory.trap.TrapInputStream.read(TrapInputStream.java:60)
com.semmle.inmemory.trap.TrapScanner.fill(TrapScanner.java:451)
com.semmle.inmemory.trap.TrapScanner.ensureNext(TrapScanner.java:428)
com.semmle.inmemory.trap.TrapScanner.nextToken(TrapScanner.java:61)
com.semmle.inmemory.trap.TRAPReader.scanTuplesAndLabels(TRAPReader.java:493)
com.semmle.inmemory.trap.TRAPLinker$TrapLinkDirectiveScanner.scanTuplesAndLabels(TRAPLinker.java:311)
com.semmle.inmemory.trap.TRAPReader.importTuples(TRAPReader.java:414)
com.semmle.inmemory.trap.TRAPReader.importTuples(TRAPReader.java:400)
com.semmle.inmemory.trap.TRAPLinker.lambda$getTasks$2(TRAPLinker.java:215)
com.semmle.util.concurrent.FutureUtils.lambda$mapAsync_$8(FutureUtils.java:161)
java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source)
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.base/java.lang.Thread.run(Unknown Source)
at (start of line)
```
I can build mozilla separately following similar steps, but when I try to create a database it fails. The database is created in the end, but some of the files are missing and are not added to the database.
Can someone please point me to the right direction on how to solve this issue?
貢獻指南
研究方向
Start by reproducing the failure with ./mach configure followed by the CodeQL database create command using ./mach build. Review the linked creation and import logs, focusing on the corrupted .trap.br files and the trap importing stage; done means the database imports successfully without missing files.
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- cpp
- 領域
- security, tooling
- Issue 類型
- 缺陷
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 停滯
- 描述清晰度
- 需要釐清
- 新手友好度
- 30/100