Sentry MSTeams Integration not reporting all alert notifications
- Dominant language
- Python
- Stars
- 44.8k
- Forks
- 4.9k
- Avg merge
- 22h 21m
- Merged PRs (30d)
- 586
Description
### Environment
self-hosted ([https://develop.sentry.dev/self-hosted/]())
### Steps to Reproduce
We recently upgraded from 23.3.1 to 24.8.0 and we had to reconfigure Microsoft Teams integration, and the issue started to happen.
We have alerts configured to send notifications to Microsoft Teams when a new issue is created with certain message content filters.
Action Interval is 30mins.
The specific event expected to be reported to Microsoft Teams shows under the Alert as triggering it, eg:

### Expected Result
We expected to receive an MS Teams message notification.
### Actual Result
We did not receive a notification, only for that specific event id.
MS Teams notifications were being received +-1h time's notifications were received. Just this one was not.
This is happening occasionally now and seems to be quite random. I was not able to find any commonality between the specific events not being reported (although triggering the alert)
The only logs related to MS Teams I was able to find in the web container are the following:
```less
web-1 | 2024-09-12T18:13:02.258835526Z 18:13:02 [INFO] sentry.integrations.msteams.webhooks: sentry.integrations.msteams.webhook (request_data={'type': 'message', 'timestamp': '2024-09-12T18:12:57.929Z', 'localTimestamp': '2024-09-12T14:12:57.929-04:00', 'id': 'f:af304f7e-6e8d-30c5-9550-2f4d820b0a2d', 'channelId': 'msteams', 'serviceUrl': 'https://smba.trafficmanager.net/amer/', 'from': {'id': '29:1h62x77I92gxZHXAn9WZACSU3xwyGpSl9nw8OROHvMd*****_3BCUGNu9Ulyyz73alw2nlX7MEB93y24_9wOO2Q', 'name': '*****', 'aadObjectId': 'c1c384ab-fbf1-4aff-9e4b-4c2824f*****'}, 'conversation': {'isGroup': True, 'conversationType': 'channel', 'tenantId': 'a86dc674-e6a2-4c25-b4ba-e36c313*****', 'id': '19:aace3adaa78044cb9ae7f23336a*****@thread.skype;messageid=1726155083650', 'name': 'Channel Name'}, 'recipient': {'id': '28:c32f9241-2c75-4524-a383-fd47fd6*****', 'name': 'Sentry msteams bot'}, 'entities': [{'locale': 'en-US', 'country': 'US', 'platform': 'Windows', 'timezone': 'America/New_York', 'type': 'clientInfo'}], 'channelData': {'channel': {'id': '19:aace3adaa78044cb9ae7f23336a*****@thread.skype'}, 'team': {'id': '19:9cfc0442beea4a7f96f01cf0ce7*****@thread.skype'}, 'tenant': {'id': 'a86dc674-e6a2-4c25-b4ba-e36c313*****'}, 'source': {'name': 'message'}, 'legacy': {'replyToId': '1:1tH-MfSjHL0K55EI1f56Y3XJCydI0AN4odBOKn*****'}}, 'replyToId': '1726155083650', 'value': {'payload': {'actionType': '1', 'groupId': 289, 'eventId': '5ffc6c60c8ee4d8da0e015558851a0c4', 'rules': [35], 'integrationId': 7}, 'resolveInput': 'resolved'}, 'locale': 'en-US', 'localTimezone': 'America/New_York'})
web-1 | 2024-09-12T18:13:11.014849387Z 18:13:11 [INFO] sentry.integrations.msteams.webhooks: sentry.integrations.msteams.webhook (request_data={'type': 'message', 'timestamp': '2024-09-12T18:13:06.949Z', 'localTimestamp': '2024-09-12T14:13:06.949-04:00', 'id': 'f:658f5e70-bbcc-e1e7-c6fb-d0bc6ca4fb89', 'channelId': 'msteams', 'serviceUrl': 'https://smba.trafficmanager.net/amer/', 'from': {'id': '29:1h62x77I92gxZHXAn9WZACSU3xwyGpSl9nw8OROHvMd*****_3BCUGNu9Ulyyz73alw2nlX7MEB93y24_9wOO2Q', 'name': '*****', 'aadObjectId': 'c1c384ab-fbf1-4aff-9e4b-4c2824f*****'}, 'conversation': {'isGroup': True, 'conversationType': 'channel', 'tenantId': 'a86dc674-e6a2-4c25-b4ba-e36c313*****', 'id': '19:aace3adaa78044cb9ae7f23336a*****@thread.skype;messageid=1726156903285', 'name': 'Channel Name'}, 'recipient': {'id': '28:c32f9241-2c75-4524-a383-fd47fd6*****', 'name': 'Sentry msteams bot'}, 'entities': [{'locale': 'en-US', 'country': 'US', 'platform': 'Windows', 'timezone': 'America/New_York', 'type': 'clientInfo'}], 'channelData': {'channel': {'id': '19:aace3adaa78044cb9ae7f23336a*****@thread.skype'}, 'team': {'id': '19:9cfc0442beea4a7f96f01cf0ce7*****@thread.skype'}, 'tenant': {'id': 'a86dc674-e6a2-4c25-b4ba-e36c313*****'}, 'source': {'name': 'message'}, 'legacy': {'replyToId': '1:1g1APO8NMTGr1NTXiXdv1JFqiWCMGfNwMjL2iB*****'}}, 'replyToId': '1726156903285', 'value': {'payload': {'actionType': '1', 'groupId': 290, 'eventId': '6cc8e54b890f46a5bf32277920fedd99', 'rules': [35], 'integrationId': 7}, 'resolveInput': 'resolved'}, 'locale': 'en-US', 'localTimezone': 'America/New_York'})
web-1 | 2024-09-12T19:17:04.409053933Z 19:17:04 [INFO] sentry.integrations.msteams.webhooks: sentry.integrations.msteams.webhook (request_data={'type': 'message', 'timestamp': '2024-09-12T19:17:01.666Z', 'localTimestamp': '2024-09-12T15:17:01.666-04:00', 'id': 'f:a8e905e0-c946-07d9-418d-0b8ff08cc4c7', 'channelId': 'msteams', 'serviceUrl': 'https://smba.trafficmanager.net/amer/', 'from': {'id': '29:1h62x77I92gxZHXAn9WZACSU3xwyGpSl9nw8OROHvMd*****_3BCUGNu9Ulyyz73alw2nlX7MEB93y24_9wOO2Q', 'name': '*****', 'aadObjectId': 'c1c384ab-fbf1-4aff-9e4b-4c2824f*****'}, 'conversation': {'isGroup': True, 'conversationType': 'channel', 'tenantId': 'a86dc674-e6a2-4c25-b4ba-e36c313*****', 'id': '19:aace3adaa78044cb9ae7f23336a*****@thread.skype;messageid=1726168329349', 'name': 'Channel Name'}, 'recipient': {'id': '28:c32f9241-2c75-4524-a383-fd47fd6*****', 'name': 'Sentry msteams bot'}, 'entities': [{'locale': 'en-US', 'country': 'US', 'platform': 'Windows', 'timezone': 'America/New_York', 'type': 'clientInfo'}], 'channelData': {'channel': {'id': '19:aace3adaa78044cb9ae7f23336a*****@thread.skype'}, 'team': {'id': '19:9cfc0442beea4a7f96f01cf0ce7*****@thread.skype'}, 'tenant': {'id': 'a86dc674-e6a2-4c25-b4ba-e36c313*****'}, 'source': {'name': 'message'}, 'legacy': {'replyToId': '1:1nRtx-MrjnQrqG43GWy52y1E8qNjYkG6z0UifT*****'}}, 'replyToId': '1726168329349', 'value': {'payload': {'actionType': '1', 'groupId': 302, 'eventId': '1d90e0589f124a5bb393779d76542323', 'rules': [60], 'integrationId': 7}, 'resolveInput': 'resolved'}, 'locale': 'en-US', 'localTimezone': 'America/New_York'})
web-1 | 2024-09-13T20:14:45.439021500Z 20:14:45 [INFO] sentry.integrations.msteams.webhooks: sentry.integrations.msteams.webhook (request_data={'type': 'message', 'timestamp': '2024-09-13T20:14:40.809Z', 'localTimestamp': '2024-09-13T16:14:40.809-04:00', 'id': 'f:e887eb7f-420d-81cb-278d-ed0b8b985df7', 'channelId': 'msteams', 'serviceUrl': 'https://smba.trafficmanager.net/amer/', 'from': {'id': '29:1h62x77I92gxZHXAn9WZACSU3xwyGpSl9nw8OROHvMd*****_3BCUGNu9Ulyyz73alw2nlX7MEB93y24_9wOO2Q', 'name': '*****', 'aadObjectId': 'c1c384ab-fbf1-4aff-9e4b-4c2824f*****'}, 'conversation': {'isGroup': True, 'conversationType': 'channel', 'tenantId': 'a86dc674-e6a2-4c25-b4ba-e36c313*****', 'id': '19:aace3adaa78044cb9ae7f23336a*****@thread.skype;messageid=1726244767674', 'name': 'Channel Name'}, 'recipient': {'id': '28:c32f9241-2c75-4524-a383-fd47fd6*****', 'name': 'Sentry msteams bot'}, 'entities': [{'locale': 'en-US', 'country': 'US', 'platform': 'Windows', 'timezone': 'America/New_York', 'type': 'clientInfo'}], 'channelData': {'channel': {'id': '19:aace3adaa78044cb9ae7f23336a*****@thread.skype'}, 'team': {'id': '19:9cfc0442beea4a7f96f01cf0ce7*****@thread.skype'}, 'tenant': {'id': 'a86dc674-e6a2-4c25-b4ba-e36c313*****'}, 'source': {'name': 'message'}, 'legacy': {'replyToId': '1:1Kq3cDiJ5RR64WCrXL6rXGP49vQKTOgQXHJ1_N*****'}}, 'replyToId': '1726244767674', 'value': {'payload': {'actionType': '1', 'groupId': 380, 'eventId': '9cf848be88c44f119914f620326dbe25', 'rules': [60], 'integrationId': 7}, 'resolveInput': 'resolved'}, 'locale': 'en-US', 'localTimezone': 'America/New_York'})
```
These are the only logs related to MS Teams that I was able to find. These appear to be the logs when the issue status is updated to "Resolved".
The event in question is '1d90e0589f124a5bb393779d76542323'
After manually finding it out on the UI, and resolving it from Sentry, it seems like there was a log mentioning that the issue got updated to "Resolved". However, no MS Teams message was found.
Has this happened before? Is there a way to debug this further? Any entries in the DB might help find the root cause?
### Product Area
Settings - Integrations
### Link
*No response*
### DSN
*No response*
### Version
24.8.0
Contributor guide
Assessment
This issue has not been assessed yet.