MultiVerb, content-type negotiation, semantics and client behavior

I am getting confused by the semantics of endpoint-level and response-level content-types in `MultiVerb`, especially regarding client behavior, and I wonder whether something needs clarification here.

# Context

Let me try to lay out my thoughts in a somewhat logical manner.

## 1. `Respond` responses

The basic usage of `MultiVerb` is to declare content-types at the `MultiVerb`-level and use `Respond` to declare all possible responses. `Respond` does not accept a content-type parameter:

```haskell
data Respond (s :: Nat) (description :: Symbol) (a :: Type)
```

which is why in the server implementation we require the `a` type to be renderable as all possibly requested content-types:

```haskell
instance
(AllMimeRender cs a, KnownStatus s)
=> ResponseRender cs (Respond s desc a)
```

This is the easy case: when the response list only contains `Respond` responses, we can always honor the client-requested content-type (assuming it is a member of the `cs` list).

## 2. `RespondAs` responses

In contrast, the `RespondAs` response type constitutes an escape hatch to the strict content-negotiation mechanism and hardcodes a content-type that may not even be a member of the MultiVerb-level list:

```haskell
data RespondAs responseContentType (s :: Nat) (description :: Symbol) (a :: Type)
```

For example, the `ResponseRender` instance in the server implementation totally disregards the endpoint-level content-type list (see the lack of `AllMimeRender cs a` here) and only looks at the response-level `ct` parameter:

```haskell
instance
( KnownStatus s
, MimeRender ct a
)
=> ResponseRender cs (RespondAs (ct :: Type) s desc a)
```

My intuition is that `RespondAs` is essentially intended to be used for error cases. E.g., success paths may support `JSON` but some errors just have a plain text body.

## 3. `RespondStreaming` and content negotiation

`RespondStreaming` is similar to `RespondAs` in that it specifies its own content-type. I think this might be problematic, as streaming responses are typically *not* error-cases, and we currently cannot ensure that these endpoints correctly participate in the content-negotiation dance.

**Aside**: The problem becomes a tiny bit more complex if we try to factor in framing (not supported by mainstream `RespondStreaming`, but I actually have working code for this): should framing depend on the selected content-type? For example `NewlineFraming` might work for `JSON` but you might want `NetstringFraming` for some other binary format. How should those two features interact?

## 4. Client-side unreachable branches

Finally, the `servant-client` MultiVerb implementation does not interact well with content-types that are not declared in the `MultiVerb` list. Consider the following API:

```haskell
type UserResponses =
'[ Respond 200 "User found" User
, RespondAs PlainText 404 "Not found" Text
]

data UserResult = UserFound User | UserNotFound Text
deriving stock (Generic)
deriving (AsUnion UserResponses) via GenericAsUnion UserResponses UserResult

type UserApi = "user" :> Capture "id" Int
:> MultiVerb 'GET '[JSON] UserResponses UserResult
```

The client builds the `Accept` header solely from the endpoint-level `cs` list:

```haskell
instance ... => HasClient m (MultiVerb method cs as r) where
clientWithRoute _ _ req =
runMultiVerbRequest ...
req { ...
, requestAccept = Seq.fromList (clientAcceptList (Proxy @cs))
}
```

So the client sends `Accept: application/json`. When the server returns a 404 with `Content-Type: text/plain`, the client validates the response content-type against what it originally requested:

```haskell
unless (any (M.matches c) accept) $
throwClientError $ UnsupportedContentType c response
```

Since `text/plain` is not in `[application/json]`, the client throws `UnsupportedContentType` before even attempting to parse. The `UserResult` type promises two branches, but `UserNotFound` is effectively unreachable from the client and replaced by an exception. This makes some branches of our union "write-only" in a sense: the server can use them to specify how to answer, but the client does not get a similarly shaped response.

One way to fix this could be to have the client gather content-types from *all* response branches, not just from `cs`. This would automatically make all branches reachable without requiring API changes. But is this truly what we want ?

# Questions

- What are the intended semantics of the `MultiVerb` content-type list ?
- What are the intended semantics of the content-types specified at the response level (`RespondAs`, `RespondStreaming`) ?
- How should the two interact (if at all) ?
- What should be the client behavior in the situation highlighted above ?