CodeQL CLI may be slow to run when the codeql.zip is extracted at $HOME
- 主要语言
- CodeQL
- 星标
- 10.1k
- 派生
- 2.1k
- 平均合并
- 2 天 15 小时
- 30 天内合并 PR
- 141
描述
In the [Getting started with the CodeQL CLI](https://docs.github.com/code-security/codeql-cli/using-the-codeql-cli/getting-started-with-the-codeql-cli) documentation, it instructs the user to extract the downloaded zip archive, but doesn't advise where this should be:
> ### 2. Extract the zip archive
>
> For Linux, Windows, and macOS users (version 10.14 "Mojave", and earlier) simply extract the zip archive.
>
> [...]
>
> ### 3. Launch `codeql`
>
> Once extracted, you can run CodeQL processes by running the `codeql` executable in a couple of ways:
>
> * By executing `/codeql/codeql`, where `` is the folder where you extracted the CodeQL CLI package.
>
>[...]
If the user chooses to extract in a directory tree which may contain a very large number of files and subdirectories, such as their home directory, then upon running any `codeql` command that resolves extensions, all of these subdirectories will be searched for files with certain filename patterns, which may take a very long time.
For example (output is from running `kill -3` on the `java` process):
```
"pool-1-thread-1" #22 prio=5 os_prio=0 cpu=40821.78ms elapsed=40.94s tid=0x00007f3dec64a460 nid=0x1bd12 runnable [0x00007f3dac3e3000]
java.lang.Thread.State: RUNNABLE
at sun.nio.fs.UnixNativeDispatcher.lstat0(java.base@17.0.7/Native Method)
at sun.nio.fs.UnixNativeDispatcher.lstat(java.base@17.0.7/UnixNativeDispatcher.java:308)
at sun.nio.fs.UnixFileAttributes.get(java.base@17.0.7/UnixFileAttributes.java:72)
at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(java.base@17.0.7/UnixFileAttributeViews.java:52)
at sun.nio.fs.UnixFileSystemProvider.readAttributes(java.base@17.0.7/UnixFileSystemProvider.java:148)
at sun.nio.fs.LinuxFileSystemProvider.readAttributes(java.base@17.0.7/LinuxFileSystemProvider.java:99)
at java.nio.file.Files.readAttributes(java.base@17.0.7/Files.java:1851)
at java.nio.file.FileTreeWalker.getAttributes(java.base@17.0.7/FileTreeWalker.java:220)
at java.nio.file.FileTreeWalker.visit(java.base@17.0.7/FileTreeWalker.java:277)
at java.nio.file.FileTreeWalker.next(java.base@17.0.7/FileTreeWalker.java:374)
at java.nio.file.Files.walkFileTree(java.base@17.0.7/Files.java:2845)
at com.semmle.frontend.packs.DiskFileSystem.directChildren(DiskFileSystem.java:56)
at com.semmle.frontend.packs.glob.PathMatchTreeWalker.pushStackEntry(PathMatchTreeWalker.java:345)
at com.semmle.frontend.packs.glob.PathMatchTreeWalker.matchDirectory(PathMatchTreeWalker.java:205)
at com.semmle.frontend.packs.glob.PathMatchTreeWalker.walk(PathMatchTreeWalker.java:121)
at com.semmle.frontend.packs.glob.PathMatcher.walkFileTree(PathMatcher.java:270)
at com.semmle.frontend.packs.ScanByManifests.scanTree(ScanByManifests.java:354)
at com.semmle.frontend.packs.ScanByManifests.scanForPacksAndWorkspaces(ScanByManifests.java:154)
at com.semmle.frontend.packs.QlPackResolver.scanForPacksAndWorkspaces(QlPackResolver.java:107)
at com.semmle.frontend.packs.QlPackContainer.computePacksAndWorkspaces(QlPackContainer.java:119)
at com.semmle.frontend.packs.QlPackContainer.lambda$new$0(QlPackContainer.java:38)
at com.semmle.frontend.packs.QlPackContainer$$Lambda$145/0x0000000800d8dd70.apply(Unknown Source)
at com.semmle.api.compilation.CachedOperation.get(CachedOperation.java:91)
- locked <0x0000000591e21578> (a java.lang.Object)
at com.semmle.frontend.packs.QlPackContainer.findPacksByName(QlPackContainer.java:69)
at com.semmle.frontend.packs.QlPackSearchContext.findPacksByName(QlPackSearchContext.java:698)
at com.semmle.frontend.packs.QlPackSearchContext.findPackByName(QlPackSearchContext.java:604)
at com.semmle.frontend.packs.QlPackSearchContext.findPackByVersion(QlPackSearchContext.java:414)
at com.semmle.frontend.packs.QlPackSearchContext.lambda$findAllPackDependencies$19(QlPackSearchContext.java:509)
at com.semmle.frontend.packs.QlPackSearchContext$$Lambda$248/0x0000000800db9560.get(Unknown Source)
at java.util.Optional.orElseGet(java.base@17.0.7/Optional.java:364)
at com.semmle.frontend.packs.QlPackSearchContext.lambda$findAllPackDependencies$20(QlPackSearchContext.java:495)
at com.semmle.frontend.packs.QlPackSearchContext$$Lambda$246/0x0000000800db90d0.apply(Unknown Source)
at java.util.Optional.map(java.base@17.0.7/Optional.java:260)
at com.semmle.frontend.packs.QlPackSearchContext.findAllPackDependencies(QlPackSearchContext.java:467)
at com.semmle.cli2.resolve.ResolveExtensionsCommand.resolvePackDependencies(ResolveExtensionsCommand.java:267)
at com.semmle.cli2.resolve.ResolveExtensionsCommand.executeJSON(ResolveExtensionsCommand.java:70)
at com.semmle.cli2.resolve.ResolveExtensionsCommand.executeJSON(ResolveExtensionsCommand.java:43)
at com.semmle.cli2.picocli.SimpleJsonSubcommand.executeInsistingOnJSONWithExistingMessages(SimpleJsonSubcommand.java:130)
at com.semmle.cli2.picocli.SimpleJsonSubcommand$$Lambda$66/0x0000000800cf0a88.apply(Unknown Source)
at com.semmle.cli2.picocli.SubcommandCommon.withCompilationMessages(SubcommandCommon.java:412)
at com.semmle.cli2.picocli.SimpleJsonSubcommand.executeInsistingOnJSON(SimpleJsonSubcommand.java:119)
at com.semmle.cli2.picocli.PlumbingRunner.call(PlumbingRunner.java:53)
at com.semmle.cli2.picocli.SubcommandCommon.callPlumbingInProcess(SubcommandCommon.java:160)
at com.semmle.cli2.test.CodeqlTestDir$ExtensionValuesCache.calculateExtensions(CodeqlTestDir.java:63)
at com.semmle.cli2.test.CodeqlTestDir$ExtensionValuesCache.get(CodeqlTestDir.java:53)
- locked <0x0000000591e21a18> (a java.util.concurrent.ConcurrentHashMap)
at com.semmle.cli2.test.CodeqlTestDir.resolveExtensions(CodeqlTestDir.java:199)
at com.semmle.cli2.test.CodeqlTestDir$$Lambda$118/0x0000000800d83990.run(Unknown Source)
at java.util.concurrent.CompletableFuture$UniRun.tryFire(java.base@17.0.7/CompletableFuture.java:787)
at java.util.concurrent.CompletableFuture.postComplete(java.base@17.0.7/CompletableFuture.java:510)
at java.util.concurrent.CompletableFuture$AsyncRun.run(java.base@17.0.7/CompletableFuture.java:1810)
at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@17.0.7/ThreadPoolExecutor.java:1136)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@17.0.7/ThreadPoolExecutor.java:635)
at java.lang.Thread.run(java.base@17.0.7/Thread.java:833)
```
The root cause seems to be that method `com.semmle.cli2.resolve.ResolveExtensionsCommand.resolvePackDependencies` calls `com.semmle.frontend.packs.QlPackSearchContext.findAllPackDependencies` with `useLegacyResolution` set to true, which ends up conducting a search of the parent of the `codeql` directory.
Please could you either change this behaviour, or make the documentation clearer about where the extraction root for `codeql.zip` should be to avoid this issue?
贡献指南
调研方向
Start with the linked “Getting started with the CodeQL CLI” page and review the “Extract the zip archive” and “Launch codeql” sections. Determine whether the intended resolution is clearer extraction guidance or a CLI behavior change; the documentation should prevent extraction under large directory trees while keeping the launch instructions accurate.
由索引模型根据 Issue 内容生成。
评估
- 领域
- cli, documentation
- Issue 类型
- 缺陷
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 42/100