eclipse-jdt / eclipse-jdt/eclipse.jdt.core

ECJ parses the same files multiple times

Open
#1,592 6 comments 0 reactions 0 assignees View on GitHub
bug needinfo performance
Dominant language
Java
Stars
237
Forks
195
Avg merge
1d 10h
Merged PRs (30d)
49

Description

@Kademlia reported an OOM issue in lombok here https://github.com/projectlombok/lombok/issues/3540. After some investigation I figured out that this happens because eclipse parses the `module-info.java` and the enum for every element in that enum. Since the enum contains a lot of elements, eclipse creates a huge amount of ASTs, which seems to be very inefficient and stresses the garbage collector.

**Steps to reproduce**

1. Import https://github.com/Kademlia/buggit
2. Add an empty package in `src\test\java\org\bukkit`
3. Add a breakpoint/log output in Parser::endParse
4. Trigger code completion in `Rip.java`

If you have lombok installed you should remove/update it before the test or your eclipse will crash really fast.

After removing the empty test package eclipse no longer parses `Material.java` that often. After removing `module-info.java` the problem disapears, even if the test package is still there.

**Additional information**

Video

I created a video that prints a line for every parse call. It is a little bit laggy and crashes with OOM in the end.

[Video](https://github.com/eclipse-jdt/eclipse.jdt.core/assets/5850477/9d1f7a2b-8bfb-4277-aef1-047682cfa509)


Stacktrace for the `module-info.java` parse

```
org.eclipse.jdt.internal.codeassist.select.SelectionParser(org.eclipse.jdt.internal.compiler.parser.Parser).endParse(int) line: 11972
org.eclipse.jdt.internal.codeassist.select.SelectionParser(org.eclipse.jdt.internal.compiler.parser.Parser).parse() line: 13173
org.eclipse.jdt.internal.codeassist.select.SelectionParser(org.eclipse.jdt.internal.compiler.parser.Parser).parse(org.eclipse.jdt.internal.compiler.env.ICompilationUnit, org.eclipse.jdt.internal.compiler.CompilationResult, int, int) line: 13400
org.eclipse.jdt.internal.codeassist.select.SelectionParser.parse(org.eclipse.jdt.internal.compiler.env.ICompilationUnit, org.eclipse.jdt.internal.compiler.CompilationResult, int, int) line: 1665
org.eclipse.jdt.internal.codeassist.select.SelectionParser(org.eclipse.jdt.internal.compiler.parser.Parser).parse(org.eclipse.jdt.internal.compiler.env.ICompilationUnit, org.eclipse.jdt.internal.compiler.CompilationResult) line: 13357
org.eclipse.jdt.internal.codeassist.select.SelectionParser(org.eclipse.jdt.internal.compiler.parser.Parser).dietParse(org.eclipse.jdt.internal.compiler.env.ICompilationUnit, org.eclipse.jdt.internal.compiler.CompilationResult) line: 11743
org.eclipse.jdt.internal.codeassist.SelectionEngine(org.eclipse.jdt.internal.codeassist.impl.Engine).accept(org.eclipse.jdt.internal.compiler.env.ICompilationUnit, org.eclipse.jdt.internal.compiler.env.AccessRestriction) line: 83
org.eclipse.jdt.internal.codeassist.SelectionEngine(org.eclipse.jdt.internal.compiler.impl.ITypeRequestor).accept(org.eclipse.jdt.internal.compiler.env.IModule, org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment) line: 61
org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.getModule(char[]) line: 221
org.eclipse.jdt.internal.core.CompilationUnit(org.eclipse.jdt.internal.compiler.env.ICompilationUnit).module(org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment) line: 64
org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration.module(org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment) line: 849
org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.buildTypeBindings(org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration, org.eclipse.jdt.internal.compiler.env.AccessRestriction) line: 475
org.eclipse.jdt.internal.codeassist.SelectionEngine.selectType(char[], org.eclipse.jdt.core.IType) line: 1678
org.eclipse.jdt.internal.core.SourceType(org.eclipse.jdt.internal.core.NamedMember).resolveType(java.lang.String, org.eclipse.jdt.core.WorkingCopyOwner) line: 328
org.eclipse.jdt.internal.core.SourceType(org.eclipse.jdt.internal.core.NamedMember).resolveType(java.lang.String) line: 270
org.eclipse.jdt.internal.ui.text.ChainElementAnalyzer.getTypeFromSignature(org.eclipse.jdt.core.IJavaProject, java.lang.String, org.eclipse.jdt.core.IType) line: 292
org.eclipse.jdt.internal.ui.text.ChainElement.setReturnType(org.eclipse.jdt.core.IJavaProject, java.lang.String, org.eclipse.jdt.core.IType) line: 107
org.eclipse.jdt.internal.ui.text.ChainElement.initializeReturnType() line: 71
org.eclipse.jdt.internal.ui.text.ChainElement.(org.eclipse.jdt.core.IJavaElement, boolean) line: 55
org.eclipse.jdt.internal.ui.text.java.ChainCompletionProposalComputer.findEntrypoints() line: 126
org.eclipse.jdt.internal.ui.text.java.ChainCompletionProposalComputer.lambda$0(org.eclipse.jdt.internal.ui.text.ChainFinder, int, int, int) line: 171
org.eclipse.jdt.internal.ui.text.java.ChainCompletionProposalComputer$$Lambda$1119.0x00007f46e4dd7020.run() line: not available
java.util.concurrent.Executors$RunnableAdapter.call() line: 539
java.util.concurrent.FutureTask.run() line: 264
java.util.concurrent.ThreadPoolExecutor.runWorker(java.util.concurrent.ThreadPoolExecutor$Worker) line: 1136
java.util.concurrent.ThreadPoolExecutor$Worker.run() line: 635
java.lang.Thread.run() line: 833

```

Stacktrace for the `Material.java` parse

```
org.eclipse.jdt.internal.core.util.CommentRecorderParser(org.eclipse.jdt.internal.compiler.parser.Parser).endParse(int) line: 11972
org.eclipse.jdt.internal.core.util.CommentRecorderParser.endParse(int) line: 135
org.eclipse.jdt.internal.core.util.CommentRecorderParser(org.eclipse.jdt.internal.compiler.parser.Parser).parse() line: 13173
org.eclipse.jdt.internal.core.util.CommentRecorderParser(org.eclipse.jdt.internal.compiler.parser.Parser).parse(org.eclipse.jdt.internal.compiler.env.ICompilationUnit, org.eclipse.jdt.internal.compiler.CompilationResult, int, int) line: 13400
org.eclipse.jdt.internal.core.util.CommentRecorderParser(org.eclipse.jdt.internal.compiler.parser.Parser).parse(org.eclipse.jdt.internal.compiler.env.ICompilationUnit, org.eclipse.jdt.internal.compiler.CompilationResult) line: 13357
org.eclipse.jdt.internal.core.util.CommentRecorderParser(org.eclipse.jdt.internal.compiler.parser.Parser).dietParse(org.eclipse.jdt.internal.compiler.env.ICompilationUnit, org.eclipse.jdt.internal.compiler.CompilationResult) line: 11743
org.eclipse.jdt.core.dom.CompilationUnitResolver(org.eclipse.jdt.internal.compiler.Compiler).accept(org.eclipse.jdt.internal.compiler.env.ICompilationUnit, org.eclipse.jdt.internal.compiler.env.AccessRestriction) line: 337
org.eclipse.jdt.core.dom.CompilationUnitResolver.accept(org.eclipse.jdt.internal.compiler.env.ICompilationUnit, org.eclipse.jdt.internal.compiler.env.AccessRestriction) line: 190
org.eclipse.jdt.core.dom.CompilationUnitResolver.accept(org.eclipse.jdt.internal.compiler.env.ISourceType[], org.eclipse.jdt.internal.compiler.lookup.PackageBinding, org.eclipse.jdt.internal.compiler.env.AccessRestriction) line: 185
org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.askForType(org.eclipse.jdt.internal.compiler.lookup.PackageBinding, char[], org.eclipse.jdt.internal.compiler.lookup.ModuleBinding) line: 346
org.eclipse.jdt.internal.compiler.lookup.SplitPackageBinding(org.eclipse.jdt.internal.compiler.lookup.PackageBinding).getTypeOrPackage(char[], org.eclipse.jdt.internal.compiler.lookup.ModuleBinding, boolean) line: 276
org.eclipse.jdt.internal.compiler.lookup.ClassScope(org.eclipse.jdt.internal.compiler.lookup.Scope).getTypeOrPackage(char[][]) line: 3644
org.eclipse.jdt.internal.compiler.ClassFile.getTypeBinding(char[], org.eclipse.jdt.internal.compiler.lookup.Scope, boolean) line: 6488
org.eclipse.jdt.internal.compiler.ClassFile.traverse(org.eclipse.jdt.internal.compiler.lookup.MethodBinding, int, byte[], int, int, java.util.Map, boolean, org.eclipse.jdt.internal.compiler.lookup.Scope) line: 7327
org.eclipse.jdt.internal.compiler.ClassFile.generateStackMapTableAttribute(org.eclipse.jdt.internal.compiler.lookup.MethodBinding, int, int, int, boolean, org.eclipse.jdt.internal.compiler.lookup.Scope) line: 5329
org.eclipse.jdt.internal.compiler.ClassFile.completeCodeAttributeForSyntheticMethod(boolean, org.eclipse.jdt.internal.compiler.lookup.SyntheticMethodBinding, int, int[], org.eclipse.jdt.internal.compiler.lookup.Scope) line: 2343
org.eclipse.jdt.internal.compiler.ClassFile.addSyntheticSwitchTable(org.eclipse.jdt.internal.compiler.lookup.SyntheticMethodBinding) line: 1551
org.eclipse.jdt.internal.compiler.ClassFile.addSpecialMethods(org.eclipse.jdt.internal.compiler.ast.TypeDeclaration) line: 1126
org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.generateCode(org.eclipse.jdt.internal.compiler.ClassFile) line: 767
org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.generateCode(org.eclipse.jdt.internal.compiler.lookup.CompilationUnitScope) line: 833
org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration.generateCode() line: 413
org.eclipse.jdt.core.dom.CompilationUnitResolver(org.eclipse.jdt.internal.compiler.Compiler).process(org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration, int) line: 916
org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(org.eclipse.jdt.core.ICompilationUnit[], java.lang.String[], org.eclipse.jdt.core.dom.ASTRequestor, int, java.util.Map, org.eclipse.jdt.core.WorkingCopyOwner, int) line: 984
org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(org.eclipse.jdt.core.ICompilationUnit[], java.lang.String[], org.eclipse.jdt.core.dom.ASTRequestor, int, java.util.Map, org.eclipse.jdt.core.IJavaProject, org.eclipse.jdt.core.WorkingCopyOwner, int, org.eclipse.core.runtime.IProgressMonitor) line: 692
org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(org.eclipse.jdt.core.IJavaElement[], int, java.util.Map, org.eclipse.jdt.core.IJavaProject, org.eclipse.jdt.core.WorkingCopyOwner, int, org.eclipse.core.runtime.IProgressMonitor) line: 907
org.eclipse.jdt.core.dom.ASTParser.createBindings(org.eclipse.jdt.core.IJavaElement[], org.eclipse.core.runtime.IProgressMonitor) line: 1112
org.eclipse.jdt.internal.ui.infoviews.JavadocView.computeFieldConstantFromTypeAST(org.eclipse.jdt.core.IField, org.eclipse.core.runtime.IProgressMonitor) line: 1353
org.eclipse.jdt.internal.ui.text.java.hover.JavadocHover.getConstantValue(org.eclipse.jdt.core.IField, org.eclipse.jdt.core.ITypeRoot, org.eclipse.jface.text.IRegion) line: 992
org.eclipse.jdt.internal.ui.text.java.JavaCompletionProposal(org.eclipse.jdt.internal.ui.text.java.AbstractJavaCompletionProposal).addConstantOrDefaultValue(java.lang.StringBuilder, org.eclipse.jdt.core.IJavaElement) line: 634
org.eclipse.jdt.internal.ui.text.java.JavaCompletionProposal(org.eclipse.jdt.internal.ui.text.java.AbstractJavaCompletionProposal).getAdditionalProposalInfo(org.eclipse.core.runtime.IProgressMonitor) line: 592
org.eclipse.jface.text.contentassist.AdditionalInfoController$Timer$2$1.run(org.eclipse.core.runtime.IProgressMonitor) line: 116
org.eclipse.core.internal.jobs.Worker.run() line: 63

```

System

Eclipse Version: 2023-09 (4.29.0) Build id: 20230907-1323
Fedora 38
Java 17


Contributor guide

Open the contributing guide

Research direction

Reproduce the repeated parsing during code completion using the buggit project, the empty test package, and module-info.java, then trace the reported paths through Parser::endParse, LookupEnvironment.getModule, and ASTParser.createBindings. Done means the same files are not parsed repeatedly and the reproduction no longer produces excessive AST creation or an OOM.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.