google / google/gapid

Reduce GAPII data transfer by caching data on the host

Open
#1,317 0 comments 0 reactions 0 assignees View on GitHub
triaged
Dominant language
Go
Stars
2.2k
Forks
329
PR merge metrics
No merged PRs in 30d

Description

There are two approaches:
1) Per-resource - the obvious one. Don't send a resource if it is already known at host.
2) Stream-level/data-agnostic - Just split the TCP/IP stream in small-ish blocks and hash those. This is more tricky to implement but it would avoid sending any previously send non-resource data, it would handle partially modified resource data, and avoid sending any repeated blocks within capture. Some care is needed to pick the block boundaries (just a fixed sized block don't work as they would not match the same data range if, say, one byte is added to the start of steam on the next run). E.g. https://github.com/dsrbecky/DeltaZip/blob/master/Splitter.cs

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.