element-hq / element-hq/element-meta

Crypto: Posthog analytics for problems when sending message keys over to-device messages

Open
#2,409 1 comment 0 reactions 0 assignees View on GitHub
A-E2EE A-Telemetry T-Feature
Dominant language
No language data
Stars
112
Forks
25
Avg merge
6h 6m
Merged PRs (30d)
4

Description

There are various failure modes that can lead to problems sending to-device messages containing message keys, which will in turn lead to [UTD errors](https://github.com/vector-im/element-meta/issues/245). Currently, these are not reported in Posthog, so we lack visibility into how often they happen.

Likely root causes are the target user's homeserver being unreachable (related: https://github.com/element-hq/element-meta/issues/2154), or our own homeserver being unresponsive. More specific examples include:
* We lack a device list and the `/keys/query` request failed (or the user is on a server we backed off from)
* We failed to `/keys/claim` for a given device (or have backed off for this device)
* We couldn't send the to-device message itself.

See also #234 which covers the *receiving* side of this (and is IMHO much lower-hanging fruit).

## Question

A single sent message could result in hundreds or thousands of errors, depending on the number of devices in the room. Similarly, a single failing user could cause lots of different sent messages to have some sort of error. Should we report an event for each device for each user for each message? Or something more intelligent? What exactly are we trying to achieve with these metrics?

---

# Implementation design

Slightly tricky because the list of things we need to report on are scattered around the codebase, though it is mostly within `matrix-sdk-crypto`. I think the first step here is to define an interface in `matrix-sdk-crypto` which emits an enum of potential error codes.

We can then add a method `OlmMachine::share_room_keys_failure_stream`, which returns a `Stream`, and each time something on the list above goes wrong, we write a new entry to the stream. The stream could then be wrapped in both (Rust) `matrix-sdk` and `matrix-js-sdk`, for turning into Posthog events.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the scattered failure paths in matrix-sdk-crypto and the proposed OlmMachine::share_room_keys_failure_stream interface, then examine the matrix-sdk and matrix-js-sdk wrappers. Done means the listed key-sharing failures produce categorized stream entries that can be turned into Posthog events, with the reporting granularity decided.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript, rust
Domain
analytics, cryptography
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.