Feature Request: Private distributed lookup architecture

How to use GitHub

• Please use the 👍 reaction to show that you are interested into the same feature.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

Feature request

The current implementation of the lookup API with a centralized service is (for good reasons) limited to exact-match lookups for userIds, if not deployed in a global-scale setup. At the same time, the service is a performance bottleneck waiting to happen (due to the way lookups are implemented with retrieval of an exact key needs two queries against users and one against store, with the later being a multi value returning query).

Instead, given that the instance's name already needs to be known in the current setup, the following could be implemented:

• an endpoint in nextcloud/server
• the endpoint ingests messages of the form:

{
  "data": {
    "search_key": SHA256(USER_ID@INSTANCE, NONCE),
    "nonce": UTIME_MS,
    }
}

When receiving such a message, a server instance can then run SHA256(USER_ID@INSTANCE, NONCE) for each local userId, and compare the results to the request. If a local userId matches, the endpoint returns True, otherwise False.

Optionally, if a user explicitly allows sharing of specific data, that specific data could also be returned.

Notes

• The requirement to know the instance is not an issue given that the current implementation also relies on knowing the exact userId
• The scheme prevents user enumeration due to the use of a nonce, i.e., a remote attacker would need to generate a new hash for each lookup on each Nextcloud instance
• Replay attacks can be prevented using limits on the nonce's age
• An on-path attacker observing the exchange would have to brute-force a list of know/generated userIds to infer the plaintext userId
• The scheme distributes the load for global lookups across all instances instead of creating a single-point-of-failure on a centralized DB
• It does not (accidentally) reveal a userId to a remote instance if that userId is not or no longer present on that instance
• As the procedure has a performance overhead vs. direct lookups, it should be sensible to rate-limit this endpoint
• Even without the SHA256 around it, this would be better than the current implementation in terms of privacy (+- the on-path observer issue if not TLS is in use.)
• This mechanic would resolve much of https://github.com/nextcloud/server/issues/51335
• Note that this is a quick draft and implications should be thought through before implementing it.

Start by reviewing the current centralized lookup API and the proposed endpoint boundary in nextcloud/server, then read the linked issue 51335 for related context. Before implementation, establish the privacy, replay, rate-limiting, and performance requirements; done means an agreed architecture rather than this draft alone.