antlr / antlr/antlr4

[CPP] Cache keep growing and will not be reset while parsing multiple files, just behaves like memory leak

Open
#2,182 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
19k
Forks
3.5k
PR merge metrics
No merged PRs in 30d

Description

Reference: https://github.com/antlr/antlr4/issues/499
At first I thought it was memory leak, however I finally find out that it is a case that the "cache" will not be release and soon grow to hundreds megabytes(with keeping the reference to a lot of ATNConfig and DFAState) after parsing some files.

I come up with a way to bypass that, like this, in grammar files for both lexer and parser:
```
@lexer::members {
void cleanCache() {
_interpreter->clearDFA();
_sharedContextCache.clear();
}
}
```
and call this method every time before I parse a file.

Maybe something like this method should be provided by runtime library, since the cache members are both static private and I cannot find any document about this problem.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the C++ runtime entry points mentioned in the report: _interpreter->clearDFA() and _sharedContextCache.clear(). Compare their behavior when parsing multiple files and determine how a supported cache-reset API should expose the needed cleanup. Done means the cache can be reset through the runtime without grammar-local workarounds and the reported growth is covered by an appropriate test.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.