dotnet / dotnet/orleans

[Idea] Detect and warn about clock skew when performing silo monitoring

Open
#7,578 0 comments 0 reactions 0 assignees View on GitHub
area-silos
Dominant language
C#
Stars
10.9k
Forks
2.1k
Avg merge
14h 42m
Merged PRs (30d)
354

Description

While discussing the fixes in #7519, @benjaminpetit mentioned that we could do more to detect and warn about clock skew, eg by including timestamps in our inter-silo ping messages.

We could decide to warn if clock skew is greater than, eg, 30 seconds or perhaps one minute.

Contributor guide

Open the contributing guide

Research direction

Start by reading the fixes discussed in #7519 and locating the inter-silo ping messages used by silo monitoring. Clarify the clock-skew threshold and warning behavior before changing anything. Done means skew can be detected from ping timestamps and a warning is emitted when the agreed threshold is exceeded.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
distributed-systems, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.