eclipse-jdt / eclipse-jdt/eclipse.jdt.core
Enhance Indexer performance
- Dominant language
- Java
- Stars
- 237
- Forks
- 195
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 49
Description
I have found two issues related to indexer performance:
1. AddJarFileToIndex doesn't use zip cache. See
- https://github.com/eclipse-jdt/eclipse.jdt.core/blob/0dbe2967235238fb5375924d17a13bcd9f9f6483/org.eclipse.jdt.core/search/org/eclipse/jdt/internal/core/search/indexing/AddJarFileToIndex.java#L151
- https://github.com/eclipse-jdt/eclipse.jdt.core/blob/0dbe2967235238fb5375924d17a13bcd9f9f6483/org.eclipse.jdt.core/search/org/eclipse/jdt/internal/core/search/indexing/AddJarFileToIndex.java#L158
2. SourceIndexer creates a new JavaSearchNameEnvironment for each source file https://github.com/eclipse-jdt/eclipse.jdt.core/blob/0dbe2967235238fb5375924d17a13bcd9f9f6483/org.eclipse.jdt.core/search/org/eclipse/jdt/internal/core/search/indexing/SourceIndexer.java#L186
We would need to create one JavaSearchNameEnvironment for a project.
I have created a PR and tested it using the [spring-boot](https://github.com/spring-projects/spring-boot) project. The Rebuild Index action takes 10-20% less when using the zip and name environment cache.
I have tested the Rebuild Index action using the following patch:
```
diff --git a/org.eclipse.jdt.core/search/org/eclipse/jdt/internal/core/search/processing/JobManager.java b/org.eclipse.jdt.core/search/org/eclipse/jdt/internal/core/search/processing/JobManager.java
index 1f28bb2fa1..f46f1b1465 100644
--- a/org.eclipse.jdt.core/search/org/eclipse/jdt/internal/core/search/processing/JobManager.java
+++ b/org.eclipse.jdt.core/search/org/eclipse/jdt/internal/core/search/processing/JobManager.java
@@ -482,6 +482,7 @@ public abstract class JobManager {
}
}
this.progressJob = null;
+ long start = 0;
while (getProcessingThread() != null) {
try {
IJob job;
@@ -513,6 +514,7 @@ public abstract class JobManager {
if (cacheZipFiles) {
JavaModelManager.getJavaModelManager().flushZipFiles(this);
cacheZipFiles = false;
+ System.out.println("RebuildIndex Took: " + (System.currentTimeMillis() - start ));
}
// just woke up, delay before processing any new jobs, allow some time for the active thread to finish
synchronized (this.idleMonitor) {
@@ -535,10 +537,17 @@ public abstract class JobManager {
this.progressJob = pJob;
}
if (!cacheZipFiles) {
+ start = System.currentTimeMillis();
JavaModelManager.getJavaModelManager().cacheZipFiles(this);
cacheZipFiles = true;
}
+ long s1 = System.currentTimeMillis();
+ // System.out.println("Job: " + job + " Start ");
job.execute(null); // may enqueue a new job
+ long took = System.currentTimeMillis() - s1;
+ if (took > 2000) {
+ System.out.println("Job: " + job + " Took: " + took);
+ }
} finally {
this.executing = false;
if (VERBOSE) {
```
The related issue - https://github.com/eclipse-jdt/eclipse.jdt.core/issues/2501 @jukzi The PR reduces the number of source file reads.
Contributor guide
Research direction
Start with AddJarFileToIndex.java at the linked lines and SourceIndexer.java around line 186, then trace the related JobManager cache flow. Review issue 2501 and the existing performance patch, and measure Rebuild Index on a representative project such as spring-boot. Done means the zip and JavaSearchNameEnvironment caches reduce redundant reads without changing indexing behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- devtools, performance
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100