eclipse-jdt / eclipse-jdt/eclipse.jdt.core

Enhance Indexer performance

Open
#4,444 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
237
Forks
195
Avg merge
1d 10h
Merged PRs (30d)
49

Description

I have found two issues related to indexer performance:

1. AddJarFileToIndex doesn't use zip cache. See
- https://github.com/eclipse-jdt/eclipse.jdt.core/blob/0dbe2967235238fb5375924d17a13bcd9f9f6483/org.eclipse.jdt.core/search/org/eclipse/jdt/internal/core/search/indexing/AddJarFileToIndex.java#L151
- https://github.com/eclipse-jdt/eclipse.jdt.core/blob/0dbe2967235238fb5375924d17a13bcd9f9f6483/org.eclipse.jdt.core/search/org/eclipse/jdt/internal/core/search/indexing/AddJarFileToIndex.java#L158

2. SourceIndexer creates a new JavaSearchNameEnvironment for each source file https://github.com/eclipse-jdt/eclipse.jdt.core/blob/0dbe2967235238fb5375924d17a13bcd9f9f6483/org.eclipse.jdt.core/search/org/eclipse/jdt/internal/core/search/indexing/SourceIndexer.java#L186
We would need to create one JavaSearchNameEnvironment for a project.

I have created a PR and tested it using the [spring-boot](https://github.com/spring-projects/spring-boot) project. The Rebuild Index action takes 10-20% less when using the zip and name environment cache.

I have tested the Rebuild Index action using the following patch:
```
diff --git a/org.eclipse.jdt.core/search/org/eclipse/jdt/internal/core/search/processing/JobManager.java b/org.eclipse.jdt.core/search/org/eclipse/jdt/internal/core/search/processing/JobManager.java
index 1f28bb2fa1..f46f1b1465 100644
--- a/org.eclipse.jdt.core/search/org/eclipse/jdt/internal/core/search/processing/JobManager.java
+++ b/org.eclipse.jdt.core/search/org/eclipse/jdt/internal/core/search/processing/JobManager.java
@@ -482,6 +482,7 @@ public abstract class JobManager {
}
}
this.progressJob = null;
+ long start = 0;
while (getProcessingThread() != null) {
try {
IJob job;
@@ -513,6 +514,7 @@ public abstract class JobManager {
if (cacheZipFiles) {
JavaModelManager.getJavaModelManager().flushZipFiles(this);
cacheZipFiles = false;
+ System.out.println("RebuildIndex Took: " + (System.currentTimeMillis() - start ));
}
// just woke up, delay before processing any new jobs, allow some time for the active thread to finish
synchronized (this.idleMonitor) {
@@ -535,10 +537,17 @@ public abstract class JobManager {
this.progressJob = pJob;
}
if (!cacheZipFiles) {
+ start = System.currentTimeMillis();
JavaModelManager.getJavaModelManager().cacheZipFiles(this);
cacheZipFiles = true;
}
+ long s1 = System.currentTimeMillis();
+ // System.out.println("Job: " + job + " Start ");
job.execute(null); // may enqueue a new job
+ long took = System.currentTimeMillis() - s1;
+ if (took > 2000) {
+ System.out.println("Job: " + job + " Took: " + took);
+ }
} finally {
this.executing = false;
if (VERBOSE) {

```

The related issue - https://github.com/eclipse-jdt/eclipse.jdt.core/issues/2501 @jukzi The PR reduces the number of source file reads.

Contributor guide

Open the contributing guide

Research direction

Start with AddJarFileToIndex.java at the linked lines and SourceIndexer.java around line 186, then trace the related JobManager cache flow. Review issue 2501 and the existing performance patch, and measure Rebuild Index on a representative project such as spring-boot. Done means the zip and JavaSearchNameEnvironment caches reduce redundant reads without changing indexing behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
devtools, performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.