eclipse-iceoryx / eclipse-iceoryx/iceoryx

Many tsan warnings during the message delivery , do we have a plan to eliminate it?

Open
#2,046 1 comment 0 reactions 0 assignees View on GitHub
refactoring technical debt
Dominant language
C++
Stars
2.2k
Forks
492
Avg merge
18h 57m
Merged PRs (30d)
1

Description

## Required information

**Operating system:**
Ubuntu 20.04 LTS

**Compiler version:**
GCC 7.5.0

**Eclipse iceoryx version:**
v2.90.0

**Observed result or behaviour:**
There are many tsan warnings during the message delivery process:
`==================
WARNING: ThreadSanitizer: data race (pid=174325)
Read of size 8 at 0x7f05c4085a28 by thread T7 (mutexes: write M34623):
#0 iox::rp::BaseRelativePointer::computeRawPtr() const external/iceoryx/iceoryx_hoofs/source/relocatable_pointer/base_relative_pointer.cpp:193 (libioxposh.so+0x00000019aaf8)
#1 iox::rp::RelativePointer::get() const bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_hoofs/_virtual_includes/iceoryx_hoofs/iceoryx_hoofs/internal/relocatable_pointer/relative_pointer.inl:70 (libioxposh.so+0x000000109912)
#2 iox::mepoo::SharedChunk::getChunkHeader() const external/iceoryx/iceoryx_posh/source/mepoo/shared_chunk.cpp:132 (libioxposh.so+0x00000010979b)
#3 iox::popo::ChunkQueuePopper >::tryPop() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/building_blocks/chunk_queue_popper.inl:58 (libioxposh.so+0x00000013b3c2)
#4 iox::popo::ChunkReceiver > >::tryGet() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/building_blocks/chunk_receiver.inl:78 (libioxposh.so+0x00000013af21)
#5 iox::popo::SubscriberPortUser::tryGetChunk() external/iceoryx/iceoryx_posh/source/popo/ports/subscriber_port_user.cpp:69 (libioxposh.so+0x00000013ab48)
#6 iox::popo::BaseSubscriber::takeChunk() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/base_subscriber.inl:93 (libados_cyber.so+0x00000021b4bc)
#7 iox::popo::UntypedSubscriberImpl >::take() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/untyped_subscriber_impl.inl:37 (libados_cyber.so+0x000000219457)
#8 Listener Callback Func
#9 iox::popo::internal::TranslateAndCallTypelessCallback::call(void*, void*, void (*)()) bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/notification_callback.inl:31 (libados_cyber.so+0x00000021c1df)
#10 iox::popo::internal::Event_t::executeCallback() external/iceoryx/iceoryx_posh/source/popo/listener.cpp:48 (libioxposh.so+0x00000010f258)
#11 iox::popo::ListenerImpl<1024ul>::threadLoop() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/listener.inl:175 (libioxposh.so+0x0000001106b4)
#12 void std::__invoke_impl::*)(), iox::popo::ListenerImpl<1024ul>*>(std::__invoke_memfun_deref, void (iox::popo::ListenerImpl<1024ul>::*&&)(), iox::popo::ListenerImpl<1024ul>*&&) external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:73 (libioxposh.so+0x000000111007)
#13 std::__invoke_result::*)(), iox::popo::ListenerImpl<1024ul>*>::type std::__invoke::*)(), iox::popo::ListenerImpl<1024ul>*>(void (iox::popo::ListenerImpl<1024ul>::*&&)(), iox::popo::ListenerImpl<1024ul>*&&) external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:95 (libioxposh.so+0x000000110731)
#14 decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)())) std::thread::_Invoker::*)(), iox::popo::ListenerImpl<1024ul>*> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:234 (libioxposh.so+0x000000111d18)
#15 std::thread::_Invoker::*)(), iox::popo::ListenerImpl<1024ul>*> >::operator()() external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:243 (libioxposh.so+0x000000111cb9)
#16 std::thread::_State_impl::*)(), iox::popo::ListenerImpl<1024ul>*> > >::_M_run() external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:186 (libioxposh.so+0x000000111c6e)
#17 (libstdc++.so.6+0x0000000d6de3)

Previous write of size 8 at 0x7f05c4085a28 by thread T35:
[failed to restore the stack]

SUMMARY: ThreadSanitizer: data race external/iceoryx/iceoryx_hoofs/source/relocatable_pointer/base_relative_pointer.cpp:193 in iox::rp::BaseRelativePointer::computeRawPtr() const
==================`

`==================
WARNING: ThreadSanitizer: data race (pid=174325)
Atomic write of size 8 at 0x7f05c4085a30 by thread T7 (mutexes: write M34623):
#0 __tsan_atomic64_fetch_add /root/objdir/../gcc-7.5.0/libsanitizer/tsan/tsan_interface_atomic.cc:611 (node_launcher+0x0000004ff050)
#1 std::__atomic_base::fetch_add(unsigned long, std::memory_order) external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/atomic_base.h:514 (libioxposh.so+0x000000109319)
#2 iox::mepoo::SharedChunk::incrementReferenceCounter() external/iceoryx/iceoryx_posh/source/mepoo/shared_chunk.cpp:49 (libioxposh.so+0x000000109319)
#3 iox::mepoo::SharedChunk::operator=(iox::mepoo::SharedChunk const&) external/iceoryx/iceoryx_posh/source/mepoo/shared_chunk.cpp:75 (libioxposh.so+0x0000001094e6)
#4 iox::mepoo::SharedChunk::SharedChunk(iox::mepoo::SharedChunk const&) external/iceoryx/iceoryx_posh/source/mepoo/shared_chunk.cpp:32 (libioxposh.so+0x00000010921b)
#5 void iox::cxx::optional::construct_value(iox::mepoo::SharedChunk&) bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_hoofs/_virtual_includes/iceoryx_hoofs/iceoryx_hoofs/internal/cxx/optional.inl:291 (libioxposh.so+0x000000121f7e)
#6 iox::mepoo::SharedChunk& iox::cxx::optional::emplace(iox::mepoo::SharedChunk&) bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_hoofs/_virtual_includes/iceoryx_hoofs/iceoryx_hoofs/internal/cxx/optional.inl:240 (libioxposh.so+0x000000121a4c)
#7 iox::cxx::optional iox::cxx::make_optional(iox::mepoo::SharedChunk&) bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_hoofs/_virtual_includes/iceoryx_hoofs/iceoryx_hoofs/internal/cxx/optional.inl:306 (libioxposh.so+0x000000120cd8)
#8 iox::popo::ChunkQueuePopper >::tryPop() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/building_blocks/chunk_queue_popper.inl:67 (libioxposh.so+0x00000013b475)
#9 iox::popo::ChunkReceiver > >::tryGet() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/building_blocks/chunk_receiver.inl:78 (libioxposh.so+0x00000013af21)
#10 iox::popo::SubscriberPortUser::tryGetChunk() external/iceoryx/iceoryx_posh/source/popo/ports/subscriber_port_user.cpp:69 (libioxposh.so+0x00000013ab48)
#11 iox::popo::BaseSubscriber::takeChunk() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/base_subscriber.inl:93 (libados_cyber.so+0x00000021b4bc)
#12 iox::popo::UntypedSubscriberImpl >::take() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/untyped_subscriber_impl.inl:37 (libados_cyber.so+0x000000219457)
#13 Listener Callback Func
#14 iox::popo::internal::TranslateAndCallTypelessCallback::call(void*, void*, void (*)()) bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/notification_callback.inl:31 (libados_cyber.so+0x00000021c1df)
#15 iox::popo::internal::Event_t::executeCallback() external/iceoryx/iceoryx_posh/source/popo/listener.cpp:48 (libioxposh.so+0x00000010f258)
#16 iox::popo::ListenerImpl<1024ul>::threadLoop() bazel-out/k8-dbg/bin/external/iceoryx/iceoryx_posh/_virtual_includes/iceoryx_posh_full/iceoryx_posh/internal/popo/listener.inl:175 (libioxposh.so+0x0000001106b4)
#17 void std::__invoke_impl::*)(), iox::popo::ListenerImpl<1024ul>*>(std::__invoke_memfun_deref, void (iox::popo::ListenerImpl<1024ul>::*&&)(), iox::popo::ListenerImpl<1024ul>*&&) external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:73 (libioxposh.so+0x000000111007)
#18 std::__invoke_result::*)(), iox::popo::ListenerImpl<1024ul>*>::type std::__invoke::*)(), iox::popo::ListenerImpl<1024ul>*>(void (iox::popo::ListenerImpl<1024ul>::*&&)(), iox::popo::ListenerImpl<1024ul>*&&) external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/invoke.h:95 (libioxposh.so+0x000000110731)
#19 decltype (__invoke((_S_declval<0ul>)(), (_S_declval<1ul>)())) std::thread::_Invoker::*)(), iox::popo::ListenerImpl<1024ul>*> >::_M_invoke<0ul, 1ul>(std::_Index_tuple<0ul, 1ul>) external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:234 (libioxposh.so+0x000000111d18)
#20 std::thread::_Invoker::*)(), iox::popo::ListenerImpl<1024ul>*> >::operator()() external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:243 (libioxposh.so+0x000000111cb9)
#21 std::thread::_State_impl::*)(), iox::popo::ListenerImpl<1024ul>*> > >::_M_run() external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/thread:186 (libioxposh.so+0x000000111c6e)
#22 (libstdc++.so.6+0x0000000d6de3)

Previous write of size 8 at 0x7f05c4085a30 by thread T35:
[failed to restore the stack]

SUMMARY: ThreadSanitizer: data race external/gcc7.5-x86_64-linux-gnu/bin/../lib/gcc/x86_64-pc-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/atomic_base.h:514 in std::__atomic_base::fetch_add(unsigned long, std::memory_order)
==================`

**Expected result or behaviour:**
no tsan warnings during the message delivery

**Conditions where it occurred / Performed steps:**
compile with tsan and run a cross process transmission

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the report with ThreadSanitizer during a cross-process transmission. Inspect BaseRelativePointer::computeRawPtr() in external/iceoryx/iceoryx_hoofs/source/relocatable_pointer/base_relative_pointer.cpp and SharedChunk reference-counting code in external/iceoryx/iceoryx_posh/source/mepoo/shared_chunk.cpp, then trace the ChunkQueuePopper path. Done means the relevant race warnings no longer occur under the reported scenario.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
distributed-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.