Reduce GAPII data transfer by caching data on the host
- Dominant language
- Go
- Stars
- 2.2k
- Forks
- 329
- PR merge metrics
- No merged PRs in 30d
Description
There are two approaches:
1) Per-resource - the obvious one. Don't send a resource if it is already known at host.
2) Stream-level/data-agnostic - Just split the TCP/IP stream in small-ish blocks and hash those. This is more tricky to implement but it would avoid sending any previously send non-resource data, it would handle partially modified resource data, and avoid sending any repeated blocks within capture. Some care is needed to pick the block boundaries (just a fixed sized block don't work as they would not match the same data range if, say, one byte is added to the start of steam on the next run). E.g. https://github.com/dsrbecky/DeltaZip/blob/master/Splitter.cs
Contributor guide
Assessment
This issue has not been assessed yet.