Abort conflicting transactions before commit
- Dominant language
- C++
- Stars
- 16.7k
- Forks
- 1.6k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 126
Description
Transactions that have conflicts waste server resources. Maybe you do a bunch of reads and send a bunch of mutations to a proxy, only to need to repeat all that work. Or maybe your write conflict ranges even end up in a resolver's memory if you're running multiple resolvers, so you cause subsequent transactions to abort unnecessarily.
Proposal: Add a transaction option that changes the behavior of non-snapshot reads. When this option is enabled, and if the storage server can tell that a read request would have a different result at a later version, then that read would fail with `not_committed`.
Notes:
1. Normally conflict detection only depends on read/write conflict ranges, which are not plumbed to storage servers. Storage servers would effectively have to infer write conflict ranges from mutations. This makes it behave differently from normal conflict detection, but I think this is OK since this behavior would be opt-in, and the resolver will still keep everything serializable.
2. I think this could be implemented on the storage server by obtaining a [view at the latest version](https://github.com/apple/foundationdb/blob/bb1724c3ae2eccea02408f850bde5495066cc9f4/fdbclient/VersionedMap.h#L814), and then checking the [insert version](https://github.com/apple/foundationdb/blob/bb1724c3ae2eccea02408f850bde5495066cc9f4/fdbclient/VersionedMap.h#L732) of each [ValueOrClearToRef](https://github.com/apple/foundationdb/blob/bb1724c3ae2eccea02408f850bde5495066cc9f4/fdbclient/VersionedMap.h#L556) returned. I don't think the latest version used here needs to be committed, but it needs to be at least the read version supplied in the read request. (I'm probably missing all kinds of subtle things here but I think that's the gist)
3. In addition to throwing `not_committed`, the client should add the conflict range to the special key space `\xff\xff/transaction/conflicting_keys/`
4. This would also effectively improve commit latency since retries would happen earlier
Contributor guide
Research direction
Start with fdbclient/VersionedMap.h, especially the view at the latest version, insert version, and ValueOrClearToRef references in the proposal. Trace how storage-server non-snapshot reads and client transaction conflict handling currently work. Done means an agreed design for early not_committed failures and recording the conflict range in \xff\xff/transaction/conflicting_keys/.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- databases, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100