getsentry / getsentry/sentry

Crons: OK checkins after a in_progress has timedout should not be discarded

Open
#54,865 2 comments 1 reaction 1 assignee Claimed by @evanpurkhiser View on GitHub
Feature Product Area: Crons
Dominant language
Python
Stars
44.8k
Forks
4.9k
Avg merge
1d 5h
Merged PRs (30d)
624

Description

Imagine the following scenario:
![image](https://github.com/getsentry/sentry/assets/1421724/ed0ed864-42d7-4255-aaa8-8ce965536b9a)

The following has happened

1. **12:00**: A in_progress check-in is sent
2. **01:00**: The 1 hour user-configured timeout is reached and the check-in is marked as **`timed_out`**
3. **01:15**: An `ok` check_in is sent.

In this scenario the users monitor is truly running longer than their configured timeout (The number of minutes a check-in is allowed to run before it's considered failed.).

However, we are currently completely discarding the OK check in.

### What do we want to do?

Some options

- Update the check-in duration with the **true duration**. Right now we don't do this and timeouts just have no duration. (https://github.com/getsentry/sentry/pull/67303)

- Create a new status such as `CLOSED_TIME_OUT`, which would represent a check-in that reached timeout state, but was also received a terminal check in after the fact that it was marked timed out.

In this scenario it's also worth considering what would happen if the check-in after the timeout was a `error` or even a `timed_out`.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.