apple / apple/swift-distributed-actors

TSAN: access race in MetricsTestKit.swift

Open
#787 2 comments 0 reactions 0 assignees View on GitHub
failed 💥 kind/bug
Dominant language
Swift
Stars
679
Forks
84
No merged PRs in 30d

Description

I had a couple of TSAN failures because of an access race in MetricsTestKit.swift. Looks like the lock isn't acquired soon enough when making a metrics instrument.

```
==================
WARNING: ThreadSanitizer: Swift access race (pid=7219)
Modifying access of Swift variable at 0x7b1000012068 by main thread:
#0 TestMetrics.makeRecorder(label:dimensions:aggregate:) MetricsTestKit.swift:67 (swift-distributed-actorsPackageTests:x86_64+0x14f77c4)
#1 protocol witness for MetricsFactory.makeRecorder(label:dimensions:aggregate:) in conformance TestMetrics (swift-distributed-actorsPackageTests:x86_64+0x14f911f)
#2 Recorder.__allocating_init(label:dimensions:aggregate:) Metrics.swift:100 (swift-distributed-actorsPackageTests:x86_64+0x49179e)
#3 Gauge.__allocating_init(label:dimensions:) Metrics.swift:183 (swift-distributed-actorsPackageTests:x86_64+0x492ce7)
#4 SWIM.Metrics.init(settings:) Metrics.swift:139 (swift-distributed-actorsPackageTests:x86_64+0x1ab5789)
#5 SWIM.Instance.init(settings:myself:) SWIMInstance.swift:362 (swift-distributed-actorsPackageTests:x86_64+0x1ac0880)
#6 SWIM.Instance.__allocating_init(settings:myself:) SWIMInstance.swift (swift-distributed-actorsPackageTests:x86_64+0x1abffb0)
#7 ActorMetricsSWIMActorPeerMetricsTests.test_swimPeer_pingRequest_shouldRemoteMetrics() SWIMActorPeerMetricsTests.swift:86 (swift-distributed-actorsPackageTests:x86_64+0x14f286a)
#8 @objc ActorMetricsSWIMActorPeerMetricsTests.test_swimPeer_pingRequest_shouldRemoteMetrics() (swift-distributed-actorsPackageTests:x86_64+0x14f5957)
#9 __invoking___ :3 (CoreFoundation:x86_64h+0x6456b)

Previous modifying access of Swift variable at 0x7b1000012068 by thread T58 (mutexes: write M4075975503767728, write M443):
#0 closure #1 in TestMetrics.make(label:dimensions:registry:maker:) MetricsTestKit.swift:82 (swift-distributed-actorsPackageTests:x86_64+0x14f83cf)
#1 partial apply for closure #1 in TestMetrics.make(label:dimensions:registry:maker:) (swift-distributed-actorsPackageTests:x86_64+0x1507d9d)
#2 NSLock.withLock(_:) MetricsTestKit.swift:415 (swift-distributed-actorsPackageTests:x86_64+0x14f8546)
#3 TestMetrics.make(label:dimensions:registry:maker:) MetricsTestKit.swift:80 (swift-distributed-actorsPackageTests:x86_64+0x14f7537)
#4 TestMetrics.makeRecorder(label:dimensions:aggregate:) MetricsTestKit.swift:67 (swift-distributed-actorsPackageTests:x86_64+0x14f7810)
#5 protocol witness for MetricsFactory.makeRecorder(label:dimensions:aggregate:) in conformance TestMetrics (swift-distributed-actorsPackageTests:x86_64+0x14f911f)
#6 Recorder.__allocating_init(label:dimensions:aggregate:) Metrics.swift:100 (swift-distributed-actorsPackageTests:x86_64+0x49179e)
#7 SWIM.Metrics.ShellMetrics.init(settings:) Metrics.swift:129 (swift-distributed-actorsPackageTests:x86_64+0x1ab503c)
#8 SWIM.Metrics.init(settings:) Metrics.swift:183 (swift-distributed-actorsPackageTests:x86_64+0x1ab72dc)
#9 SWIM.Instance.init(settings:myself:) SWIMInstance.swift:362 (swift-distributed-actorsPackageTests:x86_64+0x1ac0880)
#10 SWIM.Instance.__allocating_init(settings:myself:) SWIMInstance.swift (swift-distributed-actorsPackageTests:x86_64+0x1abffb0)
#11 closure #1 in static SWIMActorShell.behavior(settings:clusterRef:) SWIMActorShell.swift:49 (swift-distributed-actorsPackageTests:x86_64+0x9ec044)
#12 partial apply for closure #1 in static SWIMActorShell.behavior(settings:clusterRef:) (swift-distributed-actorsPackageTests:x86_64+0x9ec540)
#13 start0 #1 (_:depth:) in Behavior.start(context:) Behaviors.swift:762 (swift-distributed-actorsPackageTests:x86_64+0x5917e5)
#14 Behavior.start(context:) Behaviors.swift:780 (swift-distributed-actorsPackageTests:x86_64+0x58c149)
#15 Supervisor.interpretSupervised0(target:context:processingAction:nFoldFailureDepth:) Supervision.swift:467 (swift-distributed-actorsPackageTests:x86_64+0xcfd7d3)
#16 Supervisor.interpretSupervised0(target:context:processingAction:) Supervision.swift:458 (swift-distributed-actorsPackageTests:x86_64+0xcfaf84)
#17 Supervisor.startSupervised(target:context:) Supervision.swift:451 (swift-distributed-actorsPackageTests:x86_64+0xcfd1e0)
#18 ActorShell.interpretStart() ActorShell.swift:515 (swift-distributed-actorsPackageTests:x86_64+0x519435)
#19 ActorShell.interpretSystemMessage(message:) ActorShell.swift:315 (swift-distributed-actorsPackageTests:x86_64+0x517d19)
#19 ActorShell.interpretSystemMessage(message:) ActorShell.swift:315 (swift-distributed-actorsPackageTests:x86_64+0x517d19)
#20 Mailbox.mailboxRun(_:) Mailbox.swift:389 (swift-distributed-actorsPackageTests:x86_64+0xb08f17)
#21 Mailbox.run() Mailbox.swift:333 (swift-distributed-actorsPackageTests:x86_64+0xb01729)
#22 implicit closure #4 in implicit closure #3 in Mailbox.sendSystemMessage(_:file:line:) Mailbox.swift:233 (swift-distributed-actorsPackageTests:x86_64+0xb05fb9)
#23 partial apply for implicit closure #4 in implicit closure #3 in Mailbox.sendSystemMessage(_:file:line:) (swift-distributed-actorsPackageTests:x86_64+0xb14b1d)
#24 thunk for @escaping @callee_guaranteed () -> () (swift-distributed-actorsPackageTests:x86_64+0x57ca27)
#25 partial apply for thunk for @escaping @callee_guaranteed () -> () (swift-distributed-actorsPackageTests:x86_64+0xa81b6d)
#26 thunk for @escaping @callee_guaranteed () -> (@out ()) (swift-distributed-actorsPackageTests:x86_64+0x57cfe7)
#27 partial apply for thunk for @escaping @callee_guaranteed () -> (@out ()) (swift-distributed-actorsPackageTests:x86_64+0xa81dc5)
#28 closure #1 in FixedThreadPool.init(_:) FixedThreadPool.swift:75 (swift-distributed-actorsPackageTests:x86_64+0xa813b2)
#29 partial apply for closure #1 in FixedThreadPool.init(_:) (swift-distributed-actorsPackageTests:x86_64+0xa815f5)
#30 closure #1 in Thread.init(_:) Thread.swift:53 (swift-distributed-actorsPackageTests:x86_64+0xd1671a)
#31 partial apply for closure #1 in Thread.init(_:) (swift-distributed-actorsPackageTests:x86_64+0xd167fd)
#32 closure #1 in static Thread.runnerCallback.getter Thread.swift:132 (swift-distributed-actorsPackageTests:x86_64+0xd17c72)
#33 @objc closure #1 in static Thread.runnerCallback.getter (swift-distributed-actorsPackageTests:x86_64+0xd17d40)

Location is heap block of size 56 at 0x7b1000012040 allocated by main thread:
#0 __sanitizer_mz_malloc :3 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x5277a)
#1 _malloc_zone_malloc :2 (libsystem_malloc.dylib:x86_64+0x1bdfd)
#2 ActorMetricsSWIMActorPeerMetricsTests.init(invocation:) SWIMActorPeerMetricsTests.swift:25 (swift-distributed-actorsPackageTests:x86_64+0x14f5d68)
#3 @objc ActorMetricsSWIMActorPeerMetricsTests.init(invocation:) (swift-distributed-actorsPackageTests:x86_64+0x14f5f14)
#4 +[XCTestCase testCaseWithInvocation:] :2 (XCTest:x86_64+0x2e54b)

Mutex M4075975503767728 is already destroyed.

Mutex M443 (0x7b1800008890) created at:
#0 pthread_mutex_init :3 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x2e913)
#1 -[NSLock init] :2 (Foundation:x86_64+0x400d)
#2 NSLock.__allocating_init() (swift-distributed-actorsPackageTests:x86_64+0x14f667f)
#3 TestMetrics.init(verbose:) MetricsTestKit.swift:39 (swift-distributed-actorsPackageTests:x86_64+0x14f6ace)
#4 TestMetrics.__allocating_init(verbose:) MetricsTestKit.swift (swift-distributed-actorsPackageTests:x86_64+0x14f6a1e)
#5 ActorMetricsSWIMActorPeerMetricsTests.init(invocation:) SWIMActorPeerMetricsTests.swift:25 (swift-distributed-actorsPackageTests:x86_64+0x14f5d68)
#6 @objc ActorMetricsSWIMActorPeerMetricsTests.init(invocation:) (swift-distributed-actorsPackageTests:x86_64+0x14f5f14)
#7 +[XCTestCase testCaseWithInvocation:] :2 (XCTest:x86_64+0x2e54b)

Thread T58 (tid=778125, running) created by main thread at:
#0 pthread_create :3 (libclang_rt.tsan_osx_dynamic.dylib:x86_64h+0x2d8fd)
#1 Thread.init(_:) Thread.swift:62 (swift-distributed-actorsPackageTests:x86_64+0xd16346)
#2 Thread.__allocating_init(_:) Thread.swift (swift-distributed-actorsPackageTests:x86_64+0xd15f2c)
#3 FixedThreadPool.init(_:) FixedThreadPool.swift:63 (swift-distributed-actorsPackageTests:x86_64+0xa80e2e)
#4 FixedThreadPool.__allocating_init(_:) FixedThreadPool.swift (swift-distributed-actorsPackageTests:x86_64+0xa807c4)
#5 ActorSystem.init(settings:) ActorSystem.swift:184 (swift-distributed-actorsPackageTests:x86_64+0x540e92)
#6 ActorSystem.__allocating_init(settings:) ActorSystem.swift (swift-distributed-actorsPackageTests:x86_64+0x53fcd8)
#7 ActorSystem.__allocating_init(_:configuredWith:) ActorSystem.swift:150 (swift-distributed-actorsPackageTests:x86_64+0x53f863)
#8 ClusteredActorSystemsXCTestCase.setUpNode(_:_:) ClusteredActorSystemsXCTestCase.swift:63 (swift-distributed-actorsPackageTests:x86_64+0xe5012e)
#9 ActorMetricsSWIMActorPeerMetricsTests.test_swimPeer_pingRequest_shouldRemoteMetrics() SWIMActorPeerMetricsTests.swift:71 (swift-distributed-actorsPackageTests:x86_64+0x14f1dce)
#10 @objc ActorMetricsSWIMActorPeerMetricsTests.test_swimPeer_pingRequest_shouldRemoteMetrics() (swift-distributed-actorsPackageTests:x86_64+0x14f5957)
#11 __invoking___ :3 (CoreFoundation:x86_64h+0x6456b)

SUMMARY: ThreadSanitizer: Swift access race MetricsTestKit.swift:67 in TestMetrics.makeRecorder(label:dimensions:aggregate:)
```

Contributor guide

Open the contributing guide

Research direction

Start in MetricsTestKit.swift at TestMetrics.makeRecorder on line 67 and the make implementation around lines 80-82, then inspect the NSLock.withLock usage. Run ActorMetricsSWIMActorPeerMetricsTests.test_swimPeer_pingRequest_shouldRemoteMetrics() with ThreadSanitizer and confirm the reported access race no longer occurs.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
testing
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.