matrix-org / matrix-org/matrix-rust-sdk

Implement a hub/bus store to share across multiple processes instead of using the crypto store

Open
#2,472 2 comments 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
2.3k
Forks
500
Avg merge
1d 16h
Merged PRs (30d)
106

Description

In [this PR](https://github.com/matrix-org/matrix-rust-sdk/pull/2458) and [this PR](https://github.com/matrix-org/matrix-rust-sdk/pull/2440), we're making use of the crypto store database to store information that's unrelated to crypto, but because this store has some nice properties:

- it has a cross-process lock that's *safe*, in the sense that it's somewhat easy to blow up caches whenever discovering that another process has touched the database's data under our feet. We, in fact, reload the entire `OlmMachine` (that contains all crypto-related data), which is a bit of a nuclear option, and changing strategies here is the topic for another issue.
- in multi-process scenarios, the same backend configured per the main app is reused across all processes, meaning we can *coordinate* (aka lock it) and *communicate* (by storing/reading data into/from this database).
- we can both *read and write* safely from/to the database.

In both PRs, the information should've been stored in the state store. Unfortunately, while we could implement some cross-process lock for the state store, it wouldn't be safe (because we don't have any way to reload all the caches for the state store) and it wouldn't be shared by all processes in a multiprocess setup (because the way we retrieve notifications in such a case requires that we use an in-memory state DB).

While using the crypto store is a good short-term workaround that gets us to what we want to achieve, it's not sustainable in the long term, and other implementations should be preferred. In particular, a crypto store may not be present in a client that doesn't enable the feature flag, meaning they'll have issues with logging with OIDC in that case, because of a technical implementation detail they shouldn't be aware of.

Some ideas discussed with @Hywan:
- reuse Element iOS idea of a delayed buffer database; every time new data is stored in the state db, instead of writing to it immediately, we defer all writes by recording them in a separate "log" database, and when the main app gets focus back, it can read from this log database and apply those changes locally. Unfortunately, while this helps with sharing state between the two processes, it only solves the problem if the original data can only be *written to*, and doesn't require to be read. If it needs to be read, we don't have an obviously good mechanism that allows a "read-only" mode for the database. So this isn't fit for those use cases (third bullet point in the above list).
- building on that previous idea, we could have a third "hub" store that is just used to synchronize the two processes. It would contain all the cross-process locks baking data, as well as all the semi-transient data used to coordinate both processes (so the `pos` of a sliding sync, the latest session hash for a OIDC refresh). And later in the future it could contain that buffered data passed from the NSE process to the main app.

cc @poljar

Contributor guide

Open the contributing guide

Research direction

Start by reviewing PRs #2458 and #2440 and the existing crypto and state store implementations. Define the hub store around safe cross-process locking, shared reads and writes, and coordination data such as sliding-sync positions and OIDC session hashes; done means those use cases no longer depend on the crypto store.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.