bazelbuild / bazelbuild/remote-apis
V3 idea: Convert Digest.hash into a oneof for each hashing algorithm
- Dominant language
- Go
- Stars
- 445
- Forks
- 141
- PR merge metrics
- No merged PRs in 30d
Description
Consider the case that you have a storage service that supports multiple hashing algorithms. If someone downloads/uploads a blob, you can currently derive the hashing algorithm that was used by checking the length of Digest.hash. 32 hexadecimal characters? Likely MD5. 64 of them? Probably SHA-256. This allows a storage service to do integrity checking.
This approach becomes problematic if people want to use other hashing algorithms that use the same size. Even worse, modern hashing algorithms like [BLAKE3](https://github.com/BLAKE3-team/BLAKE3/) use a XOF (Extendable Output Function) where the digest length is user configurable. There is no proper way you can even derive the hashing algorithm in that case.
One way would be to simply remove the `hash` field, replacing it by a `oneof`:
```proto
message Digest {
oneof hash {
string sha256 = 1;
string blake3 = 2;
...
}
...
}
```
Unfortunately, not all Protobuf implementations (e.g., [the Go one](https://github.com/golang/protobuf/issues/395)) guarantee that `oneof` fields are serialized in a stable way. Maybe we need to simply use separate fields, emulating a `oneof` kind of construct at a higher level.
Contributor guide
Research direction
Start with the Digest message described in the issue and review how the Go implementation serializes oneof fields. Compare the proposed oneof with separate fields and define how supported algorithms and algorithm identification should work; done requires an agreed API design and compatibility plan.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- api, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100