github / github/codeql

[cpp] database create problems

Ouverte
#16,239 18 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
C++
Langage dominant
CodeQL
Étoiles
10.1k
Forks
2.1k
Merge moyen
2 j 15 h
PR mergées (30 j)
141

Description

I use command creating databases , like :
`codeql database create C:\test\codeql-database --source-root "E:\test-project-code\src" --language=cpp --command="call build_win_codeql.bat" --threads=0 --verbose --overwrite --mode=clear --min-disk-free=100000`

it built project successful,
but it takes too much time and costs too much disks . and after build it costs much more time to scanning trap file.
is it something wrong

all build-trace like :

...
...
[2024-04-16 11:42:00] [build-stdout] MSBuild version 17.9.8+b34f75857 for .NET Framework
[2024-04-16 11:42:00] [build-stdout] Build started 4/16/2024 11:42:00 AM.
...
[2024-04-16 21:04:28] [build-stdout] 4134 Warning(s)
[2024-04-16 21:04:28] [build-stdout] 0 Error(s)
[2024-04-16 21:04:28] [build-stdout] Time Elapsed 09:22:26.82
[2024-04-16 21:04:28] Plumbing command codeql database trace-command completed.
[2024-04-16 21:04:28] [PROGRESS] database create> Finalizing database at C:\test\devops-codeql-database.
[2024-04-16 21:04:28] Running plumbing command: codeql database finalize --threads=0 --mode=clear --min-disk-free=10000 --no-db-cluster -- C:\test\devops-codeql-database
[2024-04-16 21:04:28] Using pre-finalize script C:\codeql-home\codeql\cpp\tools\pre-finalize.cmd.
[2024-04-16 21:04:28] [PROGRESS] database finalize> Running pre-finalize script C:\codeql-home\codeql\cpp\tools\pre-finalize.cmd in C:\devops\p-6ac70a2931f74eb2a2452fb8f52372e1\src.
[2024-04-16 21:04:28] Running plumbing command: codeql database trace-command --working-dir=C:\devops\p-6ac70a2931f74eb2a2452fb8f52372e1\src --no-tracing --threads=0 -- C:\test\devops-codeql-database C:\codeql-home\codeql\cpp\tools\pre-finalize.cmd
[2024-04-16 21:04:28] [PROGRESS] database trace-command> Running command in C:\devops\p-6ac70a2931f74eb2a2452fb8f52372e1\src: [C:\codeql-home\codeql\cpp\tools\pre-finalize.cmd]
[2024-04-16 21:04:28] Plumbing command codeql database trace-command completed.
[2024-04-16 21:04:28] [PROGRESS] database finalize> Running TRAP import for CodeQL database at C:\test\devops-codeql-database...
[2024-04-16 21:04:28] Running plumbing command: codeql dataset import --dbscheme=C:\codeql-home\codeql\cpp\semmlecode.cpp.dbscheme --threads=0 -- C:\test\devops-codeql-database\db-cpp C:\test\devops-codeql-database\trap\cpp
[2024-04-16 21:04:28] Clearing disk cache since the version file C:\test\devops-codeql-database\db-cpp\default\cache\version does not exist
[2024-04-16 21:04:29] Tuple pool not found. Clearing relations with cached strings
[2024-04-16 21:04:29] Trimming disk cache at C:\test\devops-codeql-database\db-cpp\default\cache in mode clear.
[2024-04-16 21:04:29] Sequence stamp origin is -6195435911585769745
[2024-04-16 21:04:29] Pausing evaluation to hard-clear memory at sequence stamp o+0
[2024-04-16 21:04:29] Unpausing evaluation
[2024-04-16 21:04:29] Pausing evaluation to quickly trim disk at sequence stamp o+1
[2024-04-16 21:04:29] Unpausing evaluation
[2024-04-16 21:04:29] Pausing evaluation to zealously trim disk at sequence stamp o+2
[2024-04-16 21:04:29] Unpausing evaluation
[2024-04-16 21:04:29] Trimming completed (12ms): Purged everything.
[2024-04-16 21:04:29] Scanning for files in C:\test\devops-codeql-database\trap\cpp
[2024-04-16 21:05:01] Found 18594 files on disk containing 7859950 TRAP files (239.02 GiB)
[2024-04-16 21:05:01] [PROGRESS] dataset import> Grouping TRAP files by link target
[2024-04-16 21:11:37] [PROGRESS] dataset import> Grouping unlinked TRAP files together
[2024-04-16 21:12:11] [PROGRESS] dataset import> Scanning TRAP files
...
[2024-04-16 21:55:32] Scanning trace_log.cc.11991bd7.trap (trace_log.cc.11991bd7_0.trap.tar.br) (6891731 of 7859950)
...
[2024-04-17 00:06:20] Importing 96f07e7c3376ac5f473f4bab.trap (trace_log.cc.11991bd7_0.trap.tar.br) for no link target (6974499 of 7859950) [2024-04-17 00:06:20] [ERROR] dataset import> 9fafff7e4bf067c0c66b0ca7.trap (connection.cc.a0da4be2_0.trap.tar.br) for no link target, 38: com.semmle.util.exception.CatastrophicError: ID 94380083 is already mapped to 19293495 com.semmle.inmemory.util.DiskIdStore.append(DiskIdStore.java:63) com.semmle.inmemory.util.NonSequentialDiskPool.insert(NonSequentialDiskPool.java:105) com.semmle.inmemory.util.NonSequentialDiskPool.insertBucket(NonSequentialDiskPool.java:47) com.semmle.inmemory.util.DiskPool.getIdWithFreshness(DiskPool.java:171) com.semmle.inmemory.util.NonSequentialDiskPool.getIdWithFreshness(NonSequentialDiskPool.java:14) com.semmle.inmemory.populate.IdPool.lookupWithFreshness(IdPool.java:66) com.semmle.inmemory.trap.SynchronizedIdAllocator.getElementIdAndFreshness(SynchronizedIdAllocator.java:23) com.semmle.inmemory.trap.FreshIdAllocator.getElementId(FreshIdAllocator.java:22) com.semmle.inmemory.trap.TRAPReader$MetaStringBuilder.getElementId(TRAPReader.java:1016) com.semmle.inmemory.trap.TRAPReader.computeID(TRAPReader.java:1090) com.semmle.inmemory.trap.TRAPReader.computeID(TRAPReader.java:1085) com.semmle.inmemory.trap.TRAPReader.scanLabelKey(TRAPReader.java:829) com.semmle.inmemory.trap.TRAPReader.scanLabelValue(TRAPReader.java:801) com.semmle.inmemory.trap.TRAPReader.scanTuplesAndLabels(TRAPReader.java:505) com.semmle.inmemory.trap.TRAPReader.importTuples(TRAPReader.java:414) com.semmle.inmemory.trap.ImportTasksProcessor.process(ImportTasksProcessor.java:234) com.semmle.inmemory.trap.ImportTasksProcessor.lambda$importTrap$1(ImportTasksProcessor.java:154) com.semmle.util.concurrent.FutureUtils.lambda$mapAsync_$8(FutureUtils.java:161) java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(Unknown Source) java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) java.base/java.lang.Thread.run(Unknown Source) at (failed to read line: asked for -1 bytes which seems wrong!)
...
...
(I stopped running because it always print this error messages in different trap files)

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.