facebookresearch / facebookresearch/fairo
RGBD Compression + Streaming benchmarks
- Dominant language
- Jupyter Notebook
- Stars
- 929
- Forks
- 123
- PR merge metrics
- No merged PRs in 30d
Description
I did some work in Jan/Feb around trying to find optimal streaming performance for Depth16 (lossless) and RGB8 (lossy but frame-by-frame, i.e. no video codecs).
Before I forget and delete this work, I wanted to record it in a Github issue.
**Streaming Scenarios**
1. From Robot to Macbook over local high-speed Wifi
2. From Robot to Macbook over the internet (robot is connected via AT&T MiFi which is not exactly fast)
3. Within the Macbook localhost (over loopback TCP socket)
4. Within the Macbook as a library call (same process, no network serialization)
**Compression Methods**
For RGB-8 compression, the following methods were tried:
- jpeg
- webp
- png
For Depth-16 compression (lossless), the following methods were tried:
- zstd
- zlib
- bz2
- zip - Python-inbuilt zip compression
- npz - NumPy's zip compression
- rvl - [depth-specialized encoding](https://www.microsoft.com/en-us/research/uploads/prod/2018/09/p100-wilson.pdf)
- lz4 and lz4hc
Blosc was the main library used for zstd, zlib, bz2, lz4 and lz4hc compression.
I tried various low-level details within this regime:
- passing the data pointer directly to the socket call
- passing a PyBuffer to the socket call
**Transports**
The main transport I used for these numbers is zmq sockets, via [ZeroRPC](https://www.zerorpc.io/)
**Code**
- Client Script: https://github.com/facebookresearch/fairo/blob/rgbd_serdes_benchmarks/agents/locobot/bench_zero.py
- Server Script: https://github.com/facebookresearch/fairo/blob/rgbd_serdes_benchmarks/agents/locobot/bench_zero_server.py
- Mock Data + Compression / DeCompression code for all the methods: https://github.com/facebookresearch/fairo/blob/rgbd_serdes_benchmarks/agents/locobot/mock_data.py
**Optimal Encoding**
Out of the entire benchmark suite:
- For Depth - zstd with relatively low compression parameter consistently won out in both the practical scenarios (robot to macbook) for end-to-end latency. The higher the compression parameter, the longer it was bottlenecked in the compression function, and overall latency took a hit
- For RGB - JPG with 80% compression was good for both scenarios. WebP consumes too many processor cycles, so latency took a hit (the robot has Intel NUC). On a faster server-class processor, WebP was doing better than JPG for overall latency.
Code for these best found configs is here: https://github.com/soumith/rgbd_streamer/blob/main/data_compression.py#L5-L21
**All Benchmarked Numbers**
Provided for convenience as a Google Spreadsheet
https://docs.google.com/spreadsheets/d/1ccYGsOIMC0n-boZ94BmHc9ZJYa5_1_aCOt1FU8ghRGk
Contributor guide
Assessment
This issue has not been assessed yet.