facebookresearch / facebookresearch/fairo

RGBD Compression + Streaming benchmarks

Open
#1,309 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
929
Forks
123
PR merge metrics
No merged PRs in 30d

Description

I did some work in Jan/Feb around trying to find optimal streaming performance for Depth16 (lossless) and RGB8 (lossy but frame-by-frame, i.e. no video codecs).

Before I forget and delete this work, I wanted to record it in a Github issue.

**Streaming Scenarios**

1. From Robot to Macbook over local high-speed Wifi
2. From Robot to Macbook over the internet (robot is connected via AT&T MiFi which is not exactly fast)
3. Within the Macbook localhost (over loopback TCP socket)
4. Within the Macbook as a library call (same process, no network serialization)

**Compression Methods**

For RGB-8 compression, the following methods were tried:
- jpeg
- webp
- png

For Depth-16 compression (lossless), the following methods were tried:
- zstd
- zlib
- bz2
- zip - Python-inbuilt zip compression
- npz - NumPy's zip compression
- rvl - [depth-specialized encoding](https://www.microsoft.com/en-us/research/uploads/prod/2018/09/p100-wilson.pdf)
- lz4 and lz4hc

Blosc was the main library used for zstd, zlib, bz2, lz4 and lz4hc compression.
I tried various low-level details within this regime:
- passing the data pointer directly to the socket call
- passing a PyBuffer to the socket call

**Transports**

The main transport I used for these numbers is zmq sockets, via [ZeroRPC](https://www.zerorpc.io/)

**Code**
- Client Script: https://github.com/facebookresearch/fairo/blob/rgbd_serdes_benchmarks/agents/locobot/bench_zero.py
- Server Script: https://github.com/facebookresearch/fairo/blob/rgbd_serdes_benchmarks/agents/locobot/bench_zero_server.py
- Mock Data + Compression / DeCompression code for all the methods: https://github.com/facebookresearch/fairo/blob/rgbd_serdes_benchmarks/agents/locobot/mock_data.py

**Optimal Encoding**

Out of the entire benchmark suite:
- For Depth - zstd with relatively low compression parameter consistently won out in both the practical scenarios (robot to macbook) for end-to-end latency. The higher the compression parameter, the longer it was bottlenecked in the compression function, and overall latency took a hit
- For RGB - JPG with 80% compression was good for both scenarios. WebP consumes too many processor cycles, so latency took a hit (the robot has Intel NUC). On a faster server-class processor, WebP was doing better than JPG for overall latency.

Code for these best found configs is here: https://github.com/soumith/rgbd_streamer/blob/main/data_compression.py#L5-L21

**All Benchmarked Numbers**

Provided for convenience as a Google Spreadsheet

https://docs.google.com/spreadsheets/d/1ccYGsOIMC0n-boZ94BmHc9ZJYa5_1_aCOt1FU8ghRGk

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.