llvm / llvm/llvm-project

[clang-repl] Crash after failed incremental parse due to stale cached token annotation state

Open
#205,511 1 comment 0 reactions 0 assignees View on GitHub
clang-repl crash
Dominant language
LLVM
Stars
40.5k
Forks
18.7k
PR merge metrics
PR metrics pending

Description

Some Repros
```
anutosh491@Anutoshs-MacBook-Air bin % ./clang-repl
clang-repl> extern "C" int printf(const char*, ...);
clang-repl> void foo();
clang-repl> void foo() { printf("foo\n"); }
clang-repl> foo();
foo
clang-repl> void foo(
In file included from <<< inputs >>>:1:
input_line_5:1:6: error: variable has incomplete type 'void'
1 | void foo(
| ^
<<< inputs >>>:1:1: error: expected expression
In file included from <<< inputs >>>:1:
input_line_5:1:10: error: expected ';' after top level declarator
1 | void foo(
| ^
| ;
error: Parsing failed.
clang-repl> foo();
Assertion failed: (CachedTokens[CachedLexPos-1].getLastLoc() == Tok.getAnnotationEndLoc() && "The annotation should be until the most recent cached token"), function AnnotatePreviousCachedTokens, file PPCaching.cpp, line 152.
#0 0x0000000106aa1190 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x101e31190)
#1 0x0000000106a9ed78 llvm::sys::RunSignalHandlers() (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x101e2ed78)
#2 0x0000000106aa1dd4 SignalHandler(int, __siginfo*, void*) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x101e31dd4)
#3 0x000000018b5f7744 (/usr/lib/system/libsystem_platform.dylib+0x1804fb744)
#4 0x000000018b5ed8d8 (/usr/lib/system/libsystem_pthread.dylib+0x1804f18d8)
#5 0x000000018b4f4644 (/usr/lib/system/libsystem_c.dylib+0x1803f8644)
#6 0x000000018b4f38a0 (/usr/lib/system/libsystem_c.dylib+0x1803f78a0)
#7 0x0000000109018404 clang::Preprocessor::AnnotatePreviousCachedTokens(clang::Token const&) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x1043a8404)
#8 0x00000001083284f0 clang::Parser::TryAnnotateName(clang::CorrectionCandidateCallback*, clang::ImplicitTypenameContext) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x1036b84f0)
#9 0x000000010831c100 clang::Parser::isCXXDeclarationSpecifier(clang::ImplicitTypenameContext, clang::Parser::TPResult, bool*) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x1036ac100)
#10 0x000000010831b648 clang::Parser::isCXXSimpleDeclaration(bool) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x1036ab648)
#11 0x000000010831b3fc clang::Parser::isCXXDeclarationStatement(bool) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x1036ab3fc)
#12 0x00000001083246f4 clang::Parser::ParseExternalDeclaration(clang::ParsedAttributes&, clang::ParsedAttributes&, clang::ParsingDeclSpec*) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x1036b46f4)
#13 0x0000000108322d80 clang::Parser::ParseTopLevelDecl(clang::OpaquePtr&, clang::Sema::ModuleImportState&) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x1036b2d80)
#14 0x00000001083228ec clang::Parser::ParseFirstTopLevelDecl(clang::OpaquePtr&, clang::Sema::ModuleImportState&) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x1036b28ec)
#15 0x00000001075e4498 clang::IncrementalParser::ParseOrWrapTopLevelDecl() (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x102974498)
#16 0x00000001075e558c clang::IncrementalParser::Parse(llvm::StringRef) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x10297558c)
#17 0x00000001075ec464 clang::Interpreter::Parse(llvm::StringRef) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x10297c464)
#18 0x00000001075eb9e0 clang::Interpreter::ParseAndExecute(llvm::StringRef, clang::Value*) (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x10297b9e0)
#19 0x0000000104c7567c main (/Users/anutosh491/work/llvm-project/build-assert/bin/clang-repl+0x10000567c)
#20 0x000000018b22fe00
zsh: abort ./clang-repl
```

Contributor guide

Open the contributing guide

Research direction

Reproduce the clang-repl sequence in the issue, then inspect PPCaching.cpp around the AnnotatePreviousCachedTokens assertion at line 152 and the clang-repl parsing entry point. Trace how the failed incremental parse leaves cached token annotation state behind. Done means the failed input no longer causes a later valid foo() input to abort, with regression coverage for this sequence.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
cli, compilers
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.