antlr / antlr/antlr4

[Cpp] crash in multi-threading

Open
#2,004 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
19k
Forks
3.5k
PR merge metrics
No merged PRs in 30d

Description

I see ANTLR 4.7 CPP target crash, when running the parser in multiple threads.
It is probably in warm-up phase.
I am using a test scenario with concurrency::parallel_for() to parse multiple files in parallel. (4 threads on 4 core CPU)

I guess the critical piece of code is this, but I have no clue what is wrong and how to fix:

```
DefaultErrorStrategy::sync(Parser *recognizer) {
...
auto nextTokens = recognizer->getATN().nextTokens(s);
```

The call stacks looks like this:

Thread 1 (which crashed):
antlr4-runtime.dll!std::allocator::destroy(antlr4::misc::Interval * _Ptr) Line 607
antlr4-runtime.dll!std::allocator_traits >::destroy(std::allocator & _Al, antlr4::misc::Interval * _Ptr) Line 731
antlr4-runtime.dll!std::_Wrap_alloc >::destroy(antlr4::misc::Interval * _Ptr) Line 879
antlr4-runtime.dll!std::_Destroy_range > >(antlr4::misc::Interval * _First, antlr4::misc::Interval * _Last, std::_Wrap_alloc > & _Al, std::_Nonscalar_ptr_iterator_tag __formal) Line 82
antlr4-runtime.dll!std::_Destroy_range > >(antlr4::misc::Interval * _First, antlr4::misc::Interval * _Last, std::_Wrap_alloc > & _Al) Line 96
antlr4-runtime.dll!std::vector >::_Destroy(antlr4::misc::Interval * _First, antlr4::misc::Interval * _Last) Line 1567
antlr4-runtime.dll!std::vector >::operator=(const std::vector > & _Right) Line 968
antlr4-runtime.dll!antlr4::misc::IntervalSet::operator=(const antlr4::misc::IntervalSet & __that)
antlr4-runtime.dll!antlr4::atn::ATN::nextTokens(antlr4::atn::ATNState * s) Line 92 C++ Symbols loaded.
antlr4-runtime.dll!antlr4::DefaultErrorStrategy::sync(antlr4::Parser * recognizer) Line 103
myParser.dll ....

Thread 2:
antlr4-runtime.dll!antlr4::misc::IntervalSet::addAll(const antlr4::misc::IntervalSet & set) Line 138
antlr4-runtime.dll!antlr4::misc::IntervalSet::IntervalSet(const antlr4::misc::IntervalSet & set) Line 38
antlr4-runtime.dll!antlr4::DefaultErrorStrategy::sync(antlr4::Parser * recognizer) Line 103
myParser.dll ....

Thread 3:
antlr4-runtime.dll!antlr4::misc::IntervalSet::addAll(const antlr4::misc::IntervalSet & set) Line 138
antlr4-runtime.dll!antlr4::misc::IntervalSet::IntervalSet(const antlr4::misc::IntervalSet & set) Line 38
antlr4-runtime.dll!antlr4::DefaultErrorStrategy::sync(antlr4::Parser * recognizer) Line 103
myParser.dll ....

Thread 4:
antlr4-runtime.dll!antlr4::misc::IntervalSet::addAll(const antlr4::misc::IntervalSet & set) Line 138
antlr4-runtime.dll!antlr4::misc::IntervalSet::IntervalSet(const antlr4::misc::IntervalSet & set) Line 38
antlr4-runtime.dll!antlr4::DefaultErrorStrategy::sync(antlr4::Parser * recognizer) Line 103

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the crash with the reported C++ target scenario using concurrency::parallel_for() and multiple parser threads. Inspect DefaultErrorStrategy::sync(), especially the call to ATN::nextTokens(), alongside the antlr4-runtime.dll stack frames for IntervalSet and ATN. Done means the multi-threaded parser run no longer crashes, with the relevant behavior covered by a reproducible test.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.