dotnet / dotnet/orleans

Flaky test: SMS stream restart retains duplicate publisher

Closed
#10,503 16 comments 0 reactions 0 assignees View on GitHub
area-streaming bug Status: In PR test reliability
Dominant language
C#
Stars
10.9k
Forks
2.1k
Avg merge
13h 56m
Merged PRs (30d)
351

Description

## Description

`UnitTests.Streaming.Reliability.StreamReliabilityTests.SMS_StreamRel_AllSilosRestart_PubSubCounts` intermittently reports two publishers after restarting all silos, when one publisher is expected.

## Failure

- Workflow run: https://github.com/dotnet/orleans/actions/runs/31529937783
- Job: `93907285539` (`Azure Storage provider tests`, `net8.0`)

```text
After SendItem - PublisherCount for stream MemoryStreamProvider/StreamReliabilityNamespace/... = 2; expected 1
```

The failure occurred alongside the independently tracked `AQ_StreamRel_SiloJoins` timeout (#10452) on a PR which only changes a rebalancer test assertion. The corresponding behavior is unrelated to that PR.

## Expected behavior

After all silos restart and the producer is restored, the pub/sub registry should converge deterministically to one publisher before the test checks its count.

Contributor guide

Open the contributing guide

Research direction

Start with UnitTests.Streaming.Reliability.StreamReliabilityTests.SMS_StreamRel_AllSilosRestart_PubSubCounts and reproduce the failure in the linked Azure Storage provider workflow. Trace the all-silo restart and producer restoration path around the pub/sub registry; done means the test reliably observes one publisher after SendItem rather than intermittently reporting two.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
distributed-systems, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.