Wrong Notifications in Icinga due to discrepancies between icinga2 and icingaweb

Describe the bug

We noticed discrepancies between Icinga and Icingaweb regarding states and notifications:

While analyzing the icinga2 logfile we found out that Icinga executes the notification as a "Problem" although the WebUI shows that check should be ok
[06:08:06 +0200] information/Notification: Sending reminder 'Problem' notification 'host_xyz!Notification_xyz' for user 'Notication Contact' [06:08:06 +0200] information/Notification: Completed sending 'Problem' notification 'host_xyz!Notification_xyz' for checkable 'host_xyz!service_abc' and user 'Notification Contact' using command 'my-notification-command'.

We couldn't find out how the states, the check output etc. come into Icingaweb.
Our guess is that somewhere during the process in getting the information Icingaweb only gets some old values.

To Reproduce

We were not able to reproduce this issue.

Expected behavior

Icinga should execute a "Recovery" Notification because we tested the check command and it was OK.
It shouldn't be a Problem Notification

Your Environment

Version used (icinga2 --version): r2.14.0-1
Operating System and version: RHEL 7 (7.9)
Enabled features (icinga2 feature list): api checker graphite ido-mysql mainlog notification
Icinga Web 2 version and modules (System - About): 2.11.4
2 Master and 2 Satellites for each Master

Start by correlating the icinga2 notification log with the Icinga Web state for the affected check, using the reported two-master, two-satellite environment as context. The issue provides no reproduction steps, files, or tests; done would require reproducing the mismatch, identifying why a Problem state is used, and ensuring the resulting notification matches the recovered check state.