gRPC and Protobuf support for remote user-defined operations
- Dominant language
- C++
- Stars
- 93
- Forks
- 34
- PR merge metrics
- No merged PRs in 30d
Description
**Is your feature request related to a problem? Please describe.**
Data movement from VDMS to remote servers for a remote UDF involves multiple filesystem read/write operations, which could result in bottlenecks, especially for data-intensive use cases. Moreover, this flow results in multiple read/write operations even at the remote server end.
**Describe the solution you'd like**
We plan to update the workflow so that instead of files, we work with protobufs, reducing the file I/O operations for both VDMS and the remote server. With protobuf as the data format, gRPC would be a better alternative to HTTP. This solution would also make writing UDFs easier, as the user would not need to worry about file read/write or the format of the files.
Contributor guide
Assessment
This issue has not been assessed yet.