singleflight: consider API change to make WaitForResult and Do require a timeout

**Is your feature request related to a problem? Please describe.**
https://github.com/cockroachdb/cockroach/issues/117144 is an issue discovered during a customer incident where the usage of singleflight for the RoleMembership cache caused unavailability and a deadlock. The [postmortem](https://cockroachlabs.atlassian.net/wiki/x/rQBcxQ) has more details of the incident.

The problem is that this usage of singleflight opens a new read-only transaction that can end up blocking on the same transaction that earlier wrote to the same key in `system.role_members`. This is a deadlock scenario.

**Describe the solution you'd like**
To avoid the deadlock, the `WaitForResult` function could require the caller to specify a timeout. https://github.com/cockroachdb/cockroach/blob/5bf97e5bd7bf77203b5e98d9d08c17f57c9d9cc6/pkg/util/syncutil/singleflight/singleflight.go#L230

Similarly, `Do` could take a timeout also: https://github.com/cockroachdb/cockroach/blob/5bf97e5bd7bf77203b5e98d9d08c17f57c9d9cc6/pkg/util/syncutil/singleflight/singleflight.go#L141

These timeouts should be on the waiting goroutine, not the executing goroutine. Otherwise, there could be confusing behavior where a waiting goroutine could timeout rapidly after joining the singleflight.

**Describe alternatives you've considered**
We could just pick an arbitrary timeout and hardcode it within `Do` and `WaitForResult`.

Jira issue: CRDB-35166

Epic CRDB-60948