github / github/codeql

(WSL ubuntu)codeql with chormium create database failed

Đang mở
#13,562 4 bình luận 0 reaction 0 người được giao Xem trên GitHub
question
Ngôn ngữ chính
CodeQL
Star
10.1k
Fork
2.1k
Merge trung bình
2 ngày 15 giờ
Pull request đã merge (30 ngày)
141

Mô tả

## Introduction

Hey, This is a separate bug from https://github.com/github/codeql/issues/13552. when I try to slove the old bug. I want to see if it is a platform specific problem . so I use wsl ubuntu on windows do another test. and found this bug.

Here is my chormium version:

``` c++
18f@DESKTOP-ETKSDTV:~/chromium/src$ git log
commit 5394a50e522f99ac693b9f2195adab250ef40fe2 (HEAD, tag: 116.0.5791.0)
Author: Chrome Release Bot (LUCI)
Date: Thu May 25 03:04:37 2023 +0000

Publish DEPS for 116.0.5791.0
```

and the args.gn is simple:
``` c++
is_debug = false
```

and ubuntu version:

``` c++
18f@DESKTOP-ETKSDTV:~/chromium/src$ uname -a
Linux DESKTOP-ETKSDTV 5.15.90.1-microsoft-standard-WSL2 #1 SMP Fri Jan 27 02:56:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
```

when we compile chrome by this command:
``` c++
autoninja -C out/codeql chrome
```

now bug happened. I split it into 2 cases. the first one is normal. but second one is the bug. but I think I list the 2 cases will help us found the bug.

and I tried codeql differenct version:

``` c++
2.13.4 and 2.13.1
```

### case 00

case 00 is normal. when we finish the chrome build. we delete the file

``` c++
out/codeql/obj/content/browser/browser/input_handler.o
```

then we create codeql database by this:

``` c++
codeql database create ../../code_db/boom/ --overwrite --language=cpp --command='autoninja -C out/codeql chrome'
````

I could confirm the codeql database create successfully. here is the snip(original log is too long...):
``` c++
[2023-06-25 22:25:56] [build-stdout] [2112/2112] LINK ./chrome
Finalizing database at /home/18f/code_db/boom.
Successfully created database at /home/18f/code_db/boom.
```

we use this simple case to help us query:

``` sql
/**
* @name just for fun
*/

import cpp
import semmle.code.cpp.dataflow.new.DataFlow
import DataFlow::PathGraph

from Function func
where func.getQualifiedName().matches("%StringToGestureSourceType%")
select func
```

u could see everything works well.

![image](https://github.com/github/codeql/assets/135817924/e1a6a06d-b7db-4629-a4cb-19515077c46a)

This is a normal case, but I still offer the codeql database(you could see in boom.zip) , then you could diff with unnormal case.
[boom.zip](https://drive.google.com/file/d/1y1OlMd-hMM3PKzoQ9G1dRHztpPBq-Nft/view?usp=sharing)

### case 01

now we reproduce the unnormal case, jut compile full chrome like before. then we delete the all .o file in **out/codeql/obj/content/browser/browser**.

note that we don't need "gn gen out/codeql" again, becasue this folder only have *.o file. so it's ok(even u use "gn gen ..." it will still failed... I have tried).
``` c++
276 cd out/codeql/obj/content/browser/browser/
277 ls
278 rm *
````
then we still use this to create codeql database file:

``` c++
codeql database create ../../code_db/repro/ --overwrite --language=cpp --command='autoninja -C out/codeql chrome'
```

now this time you will see these log:

``` c++

Finalizing database at /home/18f/code_db/repro.
10654892_0.trap.br for no link target, 1: java.io.IOException: Brotli stream decoding failed
org.brotli.dec.BrotliInputStream.read(BrotliInputStream.java:151)
com.semmle.inmemory.trap.TrapInputStream.read(TrapInputStream.java:60)
com.semmle.inmemory.trap.TrapScanner.fill(TrapScanner.java:449)
com.semmle.inmemory.trap.TrapScanner.ensureNext(TrapScanner.java:426)
com.semmle.inmemory.trap.TrapScanner.nextToken(TrapScanner.java:59)
com.semmle.inmemory.trap.TRAPReader.scanTuplesAndLabels(TRAPReader.java:488)
com.semmle.inmemory.trap.TRAPReader.importTuples(TRAPReader.java:410)
com.semmle.inmemory.trap.ImportTasksProcessor.process(ImportTasksProcessor.java:190)
com.semmle.inmemory.trap.ImportTasksProcessor.lambda$importTrap$1(ImportTasksProcessor.java:146)
com.semmle.util.concurrent.FutureUtils.lambda$mapAsync_$8(FutureUtils.java:161)
java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source)
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.base/java.lang.Thread.run(Unknown Source)
Successfully created database at /home/18f/code_db/repro.

```

It means we meet some error, but the database create successfully.

But when we try still try to run this query, it will failed:

``` sql
/**
* @name just for fun
*/

import cpp
import semmle.code.cpp.dataflow.new.DataFlow
import DataFlow::PathGraph

from Function func
where func.getQualifiedName().matches("%StringToGestureSourceType%")
select func
```

In my personal views, seems it doesn't link handler file into the database. If u view the AST. u can't got nothing. but normal case(boom.zip) u could view right ast.

![image](https://github.com/github/codeql/assets/135817924/a6d55aac-9e09-4193-9295-c7c3229ab9c2)

I attahch the database . you could see the log in the database.

## my inverstigate

I inversitigate it a little, maybe I am wrong. but I still offer it maybe it will help u.

First, I suspect maybe it is because of my space is not enough. In my personal views. codeql create database. and zip it. but if it found the space is not enough before zip the database, it won't failed, it just abandon some information. and zip it could zip file.

So I use this command to inverstigate:

``` c++
watch du sh repro
```

It shows that the maximium size is 11G, but my available space is 200G. So I don't think is this reason.

Second, I found a similar case in https://github.com/github/codeql/issues/7582. But the problem is fixed. I think this is a similar case... when codeql extract some specific c++ code, it will failed... think about this, If I just recompile input_handler.o it will success, but if I recompile input_handler.o and another file. I will failed. So I think the problem is in the another file. This is too hard for me, but I think If u see the .log file. your could know what happened.

[repro.zip](https://drive.google.com/file/d/1SV_vBS5PoD36kugaqIcKP0U3DanttxyN/view?usp=sharing)

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Bắt đầu bằng cách so sánh cơ sở dữ liệu boom hoạt động thành công với repro và kiểm tra log xung quanh 10654892_0.trap.br. Tái hiện bằng CodeQL database create sử dụng autoninja -C out/codeql chrome sau khi xóa các tệp .o trong out/codeql/obj/content/browser/browser, sau đó kiểm tra input_handler.o và đầu ra trích xuất liên quan. Được xem là hoàn tất khi cơ sở dữ liệu hoàn tất mà không có Brotli decoding error và truy vấn StringToGestureSourceType trả về thông tin AST như mong đợi.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
cpp, linux, ubuntu
Lĩnh vực
devtools, operating-systems, security
Loại issue
Lỗi
Độ khó
4/5
Thời gian dự kiến
3-5 ngày
Mức độ hoạt động
Đình trệ
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
35/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.