[LLDB] crash when debuginfod download fails: `CachedFileStream was not committed`
- Dominant language
- LLVM
- Stars
- 40.5k
- Forks
- 18.7k
- PR merge metrics
- PR metrics pending
Description
### Version
- lldb: 22.1.0
### Summary
LLDB crashes when using debuginfod if the download of an artifact fails or is interrupted.
The crash happens in the destructor of `llvm::CachedFileStream`, which triggers a fatal error:
```
LLVM ERROR: CachedFileStream was not committed.
Fatal Python error: Aborted
```
This appears to occur when the debuginfod connection drops or becomes very slow while downloading a file.
### Reproduction
Reproduction is somewhat unreliable, but it consistently happens when the debuginfod server is very slow or the connection drops mid-download.
Example:
```bash
export DEBUGINFOD_URLS=http://your-laggy-server
lldb ./any-linux-binary-with-buildid
```
(lldb must be compiled with libcurl.so.4, otherwise debuginfod don't work)
Conditions that seem to trigger the issue:
- debuginfod server responds slowly?
- connection drops after only a few bytes of the file are downloaded
- artifact download does not complete successfully
### Crash Location
The crash occurs when the destructor of llvm::CachedFileStream runs.
Screenshot of the location in the debugger:
```
pwndbg> p Client.responseCode()
$10 = 200
```
So `Client` logic must better handle connection drops maybe
### Backtrack of crash:
```
pwndbg> bt
#0 __pthread_kill_implementation (threadid=281474630541664, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1 0x0000fffff7d41394 in __pthread_kill_internal (threadid=, signo=6) at pthread_kill.c:89
#2 0x0000fffff7ceba3c in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3 0x0000fffff7cd5a08 in __GI_abort () at abort.c:77
#4 0x0000fffff2880860 in llvm::report_fatal_error (Reason=..., GenCrashDiag=) at /build/source/llvm/lib/Support/ErrorHandling.cpp:135
#5 0x0000fffff28806a4 in llvm::report_fatal_error (Reason=, GenCrashDiag=) at /build/source/llvm/lib/Support/ErrorHandling.cpp:96
#6 0x0000fffff28675a8 in llvm::CachedFileStream::~CachedFileStream (this=0xffffe01a0370) at /build/source/llvm/include/llvm/Support/Caching.h:51
#7 llvm::localCache(llvm::Twine const&, llvm::Twine const&, llvm::Twine const&, std::__1::function >)>)::$_0::operator()(unsigned int, llvm::StringRef, llvm::Twine const&) const::CacheStream::~CacheStream() (
this=0xffffe01a0370) at /build/source/llvm/lib/Support/Caching.cpp:78
#8 llvm::localCache(llvm::Twine const&, llvm::Twine const&, llvm::Twine const&, std::__1::function >)>)::$_0::operator()(unsigned int, llvm::StringRef, llvm::Twine const&) const::CacheStream::~CacheStream() (
this=0xffffe01a0370) at /build/source/llvm/lib/Support/Caching.cpp:78
#9 0x0000fffff4cbe434 in std::__1::default_delete::operator()[abi:sn200100](llvm::CachedFileStream*) const (this=0xffffeb5dca68, __ptr=0x0)
at /nix/store/8v4sfpbc3maj5vsii6dc4kxh6ql6xyjn-zig-0.15.2/lib/zig/libcxx/include/__memory/unique_ptr.h:78
#10 std::__1::unique_ptr >::reset[abi:sn200100](llvm::CachedFileStream*) (this=0xffffeb5dca68,
__p=0x0) at /nix/store/8v4sfpbc3maj5vsii6dc4kxh6ql6xyjn-zig-0.15.2/lib/zig/libcxx/include/__memory/unique_ptr.h:300
#11 std::__1::unique_ptr >::~unique_ptr[abi:sn200100]() (this=0xffffeb5dca68)
at /nix/store/8v4sfpbc3maj5vsii6dc4kxh6ql6xyjn-zig-0.15.2/lib/zig/libcxx/include/__memory/unique_ptr.h:269
#12 llvm::(anonymous namespace)::StreamedHTTPResponseHandler::~StreamedHTTPResponseHandler (this=0xffffeb5dca20) at /build/source/llvm/lib/Debuginfod/Debuginfod.cpp:199
#13 llvm::getCachedOrDownloadArtifact (UniqueKey=..., UrlPath=..., CacheDirectoryPath=..., DebuginfodUrls=..., Timeout=...)
at /build/source/llvm/lib/Debuginfod/Debuginfod.cpp:320
#14 0x0000fffff27fd554 in GetFileForModule (module_spec=..., UrlBuilder=...)
at /build/source/lldb/source/Plugins/SymbolLocator/Debuginfod/SymbolLocatorDebuginfod.cpp:190
#15 0x0000fffff27fd7c8 in lldb_private::SymbolLocatorDebuginfod::LocateExecutableSymbolFile (module_spec=..., default_search_paths=...)
at /build/source/lldb/source/Plugins/SymbolLocator/Debuginfod/SymbolLocatorDebuginfod.cpp:210
#16 0x0000fffff221a660 in lldb_private::PluginManager::LocateExecutableSymbolFile (module_spec=..., default_search_paths=..., map=...)
at /build/source/lldb/source/Core/PluginManager.cpp:1553
#17 0x0000fffff27ffb64 in SymbolVendorELF::CreateInstance (module_sp=..., feedback_strm=)
at /build/source/lldb/source/Plugins/SymbolVendor/ELF/SymbolVendorELF.cpp:106
#18 0x0000fffff231a55c in lldb_private::SymbolVendor::FindPlugin (module_sp=..., feedback_strm=0x0) at /build/source/lldb/source/Symbol/SymbolVendor.cpp:35
#19 0x0000fffff21fd480 in lldb_private::Module::GetSymbolFile (this=0x15434b8, can_create=, feedback_strm=0x0)
at /build/source/lldb/source/Core/Module.cpp:1024
#20 0x0000fffff21ff820 in lldb_private::Module::PreloadSymbols (this=0x15434b8) at /build/source/lldb/source/Core/Module.cpp:1347
#21 0x0000fffff220a2b4 in llvm::unique_function::operator() (this=0x0) at /build/source/llvm/include/llvm/ADT/FunctionExtras.h:364
#22 std::__1::__invoke[abi:sn200100]>(llvm::unique_function&&) (__f=...)
at /nix/store/8v4sfpbc3maj5vsii6dc4kxh6ql6xyjn-zig-0.15.2/lib/zig/libcxx/include/__type_traits/invoke.h:179
#23 __execute<> (this=0x0) at /nix/store/8v4sfpbc3maj5vsii6dc4kxh6ql6xyjn-zig-0.15.2/lib/zig/libcxx/include/future:1858
#24 std::__1::__async_func>::operator()[abi:sn200100]() (this=0x0)
at /nix/store/8v4sfpbc3maj5vsii6dc4kxh6ql6xyjn-zig-0.15.2/lib/zig/libcxx/include/future:1852
#25 std::__1::__deferred_assoc_state > >::__execute (this=0x1520390)
at /nix/store/8v4sfpbc3maj5vsii6dc4kxh6ql6xyjn-zig-0.15.2/lib/zig/libcxx/include/future:827
#26 0x0000fffff2897e40 in llvm::unique_function::operator() (this=0xffffeb5dd770) at /build/source/llvm/include/llvm/ADT/FunctionExtras.h:364
#27 llvm::StdThreadPool::processTasks (this=this@entry=0x1134ce0, WaitingForGroup=0x0, WaitingForGroup@entry=0xffffeb5dd830)
at /build/source/llvm/lib/Support/ThreadPool.cpp:119
#28 0x0000fffff28998b8 in llvm::StdThreadPool::grow(int)::$_0::operator()() const (this=0x155f640) at /build/source/llvm/lib/Support/ThreadPool.cpp:62
#29 llvm::thread::GenericThreadProxy >(void*)::{lambda(auto:1&&, (auto:2&&)...)#1}::operator()(llvm::StdThreadPool::grow(int)::$_0&) const (F=..., this=) at /build/source/llvm/include/llvm/Support/thread.h:46
#30 std::__1::__invoke[abi:sn200100] >(void*)::{lambda(auto:1&&, (auto:2&&)...)#1}, llvm::StdThreadPool::grow(int)::$_0&>(llvm::thread::GenericThreadProxy >(void*)::{lambda(auto:1&&, (auto:2&&)...)#1}&&, llvm::StdThreadPool::grow(int)::$_0&) (__args=..., __f=...)
at /nix/store/8v4sfpbc3maj5vsii6dc4kxh6ql6xyjn-zig-0.15.2/lib/zig/libcxx/include/__type_traits/invoke.h:179
#31 std::__1::__apply_tuple_impl[abi:sn200100] >(void*)::{lambda(auto:1&&, (auto:2&&)...)#1}, std::__1::tuple&, 0ul>(llvm::thread::GenericThreadProxy >(void*)::{lambda(auto:1&&, (auto:2&&)...)#1}&&, std::__1::tuple&, std::__1::__tuple_indices<0ul>) (__t=..., __f=...)
at /nix/store/8v4sfpbc3maj5vsii6dc4kxh6ql6xyjn-zig-0.15.2/lib/zig/libcxx/include/tuple:1375
#32 std::__1::apply[abi:sn200100] >(void*)::{lambda(auto:1&&, (auto:2&&)...)#1}, std::__1::tuple&>(llvm::thread::GenericThreadProxy >(void*)::{lambda(auto:1&&, (auto:2&&)...)#1}&&, std::__1::tuple&) (__t=..., __f=...)
at /nix/store/8v4sfpbc3maj5vsii6dc4kxh6ql6xyjn-zig-0.15.2/lib/zig/libcxx/include/tuple:1379
#33 llvm::thread::GenericThreadProxy > (Ptr=0x155f640) at /build/source/llvm/include/llvm/Support/thread.h:44
#34 llvm::thread::ThreadProxy > (Ptr=0x155f640) at /build/source/llvm/include/llvm/Support/thread.h:58
#35 0x0000fffff7d3f504 in start_thread (arg=0xffffeb5de160) at pthread_create.c:448
#36 0x0000fffff7daa99c in thread_start () at ../sysdeps/unix/sysv/linux/aarch64/clone3.S:79
```
Contributor guide
Assessment
This issue has not been assessed yet.