[clang-repl] Crash after failed incremental parse due to stale cached token annotation state
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
Some Repros
```
anutosh491@Anutoshs-MacBook-Air bin % ./clang-repl
clang-repl> extern "C" int printf(const char*, ...);
clang-repl> void foo();
clang-repl> void foo() { printf("foo\n"); }
clang-repl> foo();
foo
clang-repl> void foo(
In file included from <<< inputs >>>:1:
input_line_5:1:6: error: variable has incomplete type 'void'
1 | void foo(
| ^
<<< inputs >>>:1:1: error: expected expression
In file included from <<< inputs >>>:1:
input_line_5:1:10: error: expected ';' after top level declarator
1 | void foo(
| ^
| ;
error: Parsing failed.
clang-repl> foo();
Assertion failed: (CachedTokens[CachedLexPos-1].getLastLoc() == Tok.getAnnotationEndLoc() && "The annotation should be until the most recent cached token"), function AnnotatePreviousCachedTokens, file PPCaching.cpp, line 152.
#0 0x0000000106aa1190 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x101e31190)
#1 0x0000000106a9ed78 llvm::sys::RunSignalHandlers() (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x101e2ed78)
#2 0x0000000106aa1dd4 SignalHandler(int, __siginfo*, void*) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x101e31dd4)
#3 0x000000018b5f7744 (/usr/lib/system/libsystem_platform.dylib+0x1804fb744)
#4 0x000000018b5ed8d8 (/usr/lib/system/libsystem_pthread.dylib+0x1804f18d8)
#5 0x000000018b4f4644 (/usr/lib/system/libsystem_c.dylib+0x1803f8644)
#6 0x000000018b4f38a0 (/usr/lib/system/libsystem_c.dylib+0x1803f78a0)
#7 0x0000000109018404 clang::Preprocessor::AnnotatePreviousCachedTokens(clang::Token const&) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x1043a8404)
#8 0x00000001083284f0 clang::Parser::TryAnnotateName(clang::CorrectionCandidateCallback*, clang::ImplicitTypenameContext) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x1036b84f0)
#9 0x000000010831c100 clang::Parser::isCXXDeclarationSpecifier(clang::ImplicitTypenameContext, clang::Parser::TPResult, bool*) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x1036ac100)
#10 0x000000010831b648 clang::Parser::isCXXSimpleDeclaration(bool) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x1036ab648)
#11 0x000000010831b3fc clang::Parser::isCXXDeclarationStatement(bool) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x1036ab3fc)
#12 0x00000001083246f4 clang::Parser::ParseExternalDeclaration(clang::ParsedAttributes&, clang::ParsedAttributes&, clang::ParsingDeclSpec*) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x1036b46f4)
#13 0x0000000108322d80 clang::Parser::ParseTopLevelDecl(clang::OpaquePtr&, clang::Sema::ModuleImportState&) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x1036b2d80)
#14 0x00000001083228ec clang::Parser::ParseFirstTopLevelDecl(clang::OpaquePtr&, clang::Sema::ModuleImportState&) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x1036b28ec)
#15 0x00000001075e4498 clang::IncrementalParser::ParseOrWrapTopLevelDecl() (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x102974498)
#16 0x00000001075e558c clang::IncrementalParser::Parse(llvm::StringRef) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x10297558c)
#17 0x00000001075ec464 clang::Interpreter::Parse(llvm::StringRef) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x10297c464)
#18 0x00000001075eb9e0 clang::Interpreter::ParseAndExecute(llvm::StringRef, clang::Value*) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x10297b9e0)
#19 0x0000000104c7567c main (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x10000567c)
#20 0x000000018b22fe00
zsh: abort ./clang-repl
```
Contributor guide
Research direction
Reproduce the clang-repl sequence in the issue, then inspect PPCaching.cpp around the AnnotatePreviousCachedTokens assertion at line 152 and the clang-repl parsing entry point. Trace how the failed incremental parse leaves cached token annotation state behind. Done means the failed input no longer causes a later valid foo() input to abort, with regression coverage for this sequence.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- cli, compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100