Client-Side Applications, Restricted Access and Application-Specific Tokens

This issue has been migrated from [#1194](https://github.com/matrix-org/synapse/issues/1194).

---

One particular deficit in the current Matrix Spec / Synapse implementation is the capability for non-registered users to gain read-only access to a room. There is a pretty common use case for this in terms of a client-side application built with Matrix as a messaging log backend.

For example consider an application for commenting a sports cast - only the authoring user sends writes to the room, while all other client-applications should be able to access the room with read-only permissions. In the latter case and for other similar use-cases the necessity of forcing users to register (at least they must as guests given the current spec/implementation) would hurt the usability of the client-side application. The alternative of providing all users with a mock account, even if its power-level is restricted from writing in the room in question - may still open an unnecessary window for abuse in other rooms permissive of guest access.

## Possible Solutions

1) One possibility for this kind of read-only access is allowing client-applications to request 'ephemeral access' which has no user entry associated with it. In this case the room would have to have perhaps an additional readability state (on top of 'world_readable') which would be checked when a client requests a token for reading room events without a user account (user_info retrieval would have to account for this, e.g. by providing a mock user_info object).

2) An option, which could follow from perhaps a very different requirement would be for a user to create an application specific token that would have caveats/restrictions that they could then distribute publicly. e.g. Supposing Homeservers/Synapse supported something of the application-specific-token permission model (I'm sure there is a correct term for this in the literature but I mean e.g. Github API tokens) then a user authoring the relevant content for a client-application could request a token with read-only permissions for a particular room-id -- and then distribute this token to application users.
This would be slightly more complicated than something like the 'ephemeral' token suggested above (since the room administrator would need to actively craft and distribute the tokens) but on the other-hand would be a particular use-case of a solution with other applications, rather than a solution limited to one particular problem/access restriction type.

### Ways of restricting Application Specific Tokens

The two obvious choices I guess would be explicit DB based restrictions and Macaroon caveats. Since Synapse is already Caveat based I spent a little more time exploring the later, although since Caveats are no longer explicitly spec'd application-specific tokens for user accounts could also have an open implementation (via. DB stored parameters or caveats supposing a different HS also supported granting restricted tokens).

#### Thoughts on caveats and Macaroons

In this way for example a read-only access type to room events for a specific room might be crafted by adding the following caveats to an existing token:
```
'access_method = GET'
'room_id = '
```
One thing that I'm not quite sure about is it seems that it would be necessary to restrict endpoint access to limited to a particular subset endpoints as well - particularly this is because of some weaknesses with both the current method of handling caveats and the possibility of e.g. params or other exact caveat specifications to occur both in path params (e.g. /room//) and in query params (e.g. events?room_id=).

One thing to note is that the current method of applying caveats adds verifications for caveat propositions which might not be provided by a Macaroon at all. e.g. (https://github.com/matrix-org/synapse/blob/master/synapse/api/auth.py#L812)
```python
satisfy_exact('guest = true')
```

It relies on the behavior that if the `guest =` proposition is not provided then caveat verification is ignored. The other Macaroon spec requirement that all caveats be understood is checked separately in `_verify_recognizes_caveats`; this is somewhat weak (more on this later). Therefore when implementing application-specific restrictions, since a Macaroon may not or may not contain an additional caveat caveat e.g. `room_id = `, but should always be verified with the current system, the method would have to be something like:
```python
if caveat_prop_name in request.args:
param = request.args[caveat_prop_name][0]
# In my testing I added a method to set `_kwargs` on the request when the kwargs dict is created by the HTTP Server dispatch method
# to make this kind of comparison possible
elif caveat_prop_name in request.kwargs:
param = requets._kwargs[caveat_prop_name]
else:
param = 'ANY'

verifier.satisfy_exact('%s = %s' %(caveat_var_name, param))
```

With something like this `'room_id = '` would need to match the arguments of the request, and a room_id specific caveat would always fail on 'ANY'. But the general permissiveness for query params is a disadvantage here (if permitted endpoints aren't explicitly specified in the caveat) since one could do something like `/user_settings?room_id=` which would satisfy the caveat -- but not the permission restrictions the user had in mind when they crafted an application-token with that caveat.

Some other concerns are the current unfinished transfer of all user_info to Macaroons (listed as TODO https://github.com/matrix-org/synapse/blob/master/synapse/api/auth.py#L751):

Since `_look_up_user_by_access_token` is called even after Macaroon deserialization and validation then application-specific Macaroons would fail here - so either the base token must be regenerated, by applying all of the 'base' caveats without additional ones to a newly minted token (which might have performance consequences) or a different method of user look-up / token-expiration checking should be chosen.

Another concern with application-specific tokens and caveats is refresh_token, since refreshing the base token should in theory render all other tokens generated from the previous one invalid. In this case it might make sense for refresh to update only expiry and invalidate other short-lived tokens but not the base token (or otherwise provide a separate base-token for crafting application-specific tokens with additional caveats).

One alternative to the pre-emptive caveat verification method currently used by Synapse, that is the process of adding `satisfy_exact` statements for caveats which might or might not exist, would be something of a regex matcher for `' = '` type propositions (similar to the way URL's are currently routed), each caveat provided by a Macaroon would have to be matched by at-least one pattern, which would then provide a verifier class for determining the the caveats validity. e.g. for room_id the verifier class might be something like:

```python
class RoomIdCaveatVerifier:
PATTERN = "(?Proom_id) = (?P![\w]+:[\w\-]+)"

def __init__(self, caveat, context):
# initialize with caveat and context

def verify(self):
# extract request - room_id from context check against
```

A structure like this would also supercede the what seems to me like a somewhat vulnerable separation of concerns created by `_verify_recognizes_caveats` being a separate check from Synapse's ability to actually recognize them. Also it would be more extensible where caveat propositions required to do somewhat broader logic at any point e.g. `' IN {...}'`