apache / apache/arrow

[C++] Valgrind failure in arrow-gcsfs-test

Open
#33,379 0 comments 0 reactions 0 assignees View on GitHub
Component: C++ Priority: Critical Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

https://dev.azure.com/ursacomputing/crossbow/_build/results?buildId=38546&view=logs&j=0da5d1d9-276d-5173-c4c4-9d4d4ed14fdb&t=d9b15392-e4ce-5e4c-0c8c-b69645229181

```

==11267==
==11267== HEAP SUMMARY:
==11267== in use at exit: 12,091 bytes in 190 blocks
==11267== total heap usage: 982,685 allocs, 982,495 frees, 1,332,264,705 bytes allocated
==11267==
==11267== 192 bytes in 8 blocks are definitely lost in loss record 35 of 45
==11267== at 0x40377A5: operator new(unsigned long, std::nothrow_t const&) (vg_replace_malloc.c:542)
==11267== by 0x682B079: __cxa_thread_atexit (atexit_thread.cc:152)
==11267== by 0x672F2D6: google::cloud::v2_3_0::internal::OptionsSpan::OptionsSpan(google::cloud::v2_3_0::Options) (in /opt/conda/envs/arrow/lib/libgoogle_cloud_cpp_common.so.2.3.0)
==11267== by 0x5DFCA33: google::cloud::v2_3_0::Status google::cloud::storage::v2_3_0::Client::DeleteObject(std::__cxx11::basic_string, std::allocator > const&, std::__cxx11::basic_string, std::allocator > const&, google::cloud::storage::v2_3_0::Generation&&) (client.h:1285)
==11267== by 0x5DFD022: operator() (gcsfs.cc:550)
==11267== by 0x5DFD022: operator()&)>&, google::cloud::v2_3_0::StatusOr&> (future.h:150)
==11267== by 0x5DFD022: __invoke_impl&, arrow::fs::GcsFileSystem::Impl::DeleteDirContents(const arrow::fs::(anonymous namespace)::GcsPath&, bool, const arrow::io::IOContext&)::&)>&, google::cloud::v2_3_0::StatusOr&> (invoke.h:60)
==11267== by 0x5DFD022: __invoke&, arrow::fs::GcsFileSystem::Impl::DeleteDirContents(const arrow::fs::(anonymous namespace)::GcsPath&, bool, const arrow::io::IOContext&)::&)>&, google::cloud::v2_3_0::StatusOr&> (invoke.h:95)
==11267== by 0x5DFD022: __call (functional:416)
==11267== by 0x5DFD022: operator()<> (functional:499)
==11267== by 0x5DFD022: arrow::internal::FnOnce::FnImpl, arrow::fs::GcsFileSystem::Impl::DeleteDirContents(arrow::fs::(anonymous namespace)::GcsPath const&, bool, arrow::io::IOContext const&)::{lambda(google::cloud::v2_3_0::StatusOr const&)#1}, google::cloud::v2_3_0::StatusOr)> >::invoke() (functional.h:152)
==11267== by 0x50BDAA1: operator() (functional.h:140)
==11267== by 0x50BDAA1: arrow::internal::WorkerLoop(std::shared_ptr, std::_List_iterator) (thread_pool.cc:243)
==11267== by 0x50BE161: operator() (thread_pool.cc:414)
==11267== by 0x50BE161: __invoke_impl > (invoke.h:60)
==11267== by 0x50BE161: __invoke > (invoke.h:95)
==11267== by 0x50BE161: _M_invoke<0> (thread:264)
==11267== by 0x50BE161: operator() (thread:271)
==11267== by 0x50BE161: std::thread::_State_impl > >::_M_run() (thread:215)
==11267== by 0x6849A92: execute_native_thread_routine (thread.cc:82)
==11267== by 0x69666DA: start_thread (pthread_create.c:463)
==11267== by 0x6C9F61E: clone (clone.S:95)
==11267==
{

Memcheck:Leak
match-leak-kinds: definite
fun:_ZnwmRKSt9nothrow_t
fun:execute_native_thread_routine
fun:start_thread
fun:clone
}
```

**Reporter**: [David Li](https://issues.apache.org/jira/browse/ARROW-18191) / @lidavidm

**Note**: *This issue was originally created as [ARROW-18191](https://issues.apache.org/jira/browse/ARROW-18191). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start with the linked Azure Valgrind log and reproduce the arrow-gcsfs-test failure. Inspect gcsfs.cc at line 550, especially DeleteDirContents, and trace the google-cloud OptionsSpan allocation shown in the stack. Done means the test no longer reports this definite leak or the issue is documented as an external-library limitation with an appropriate suppression.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, google-cloud
Domain
cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.