IntelLabs / IntelLabs/vdms

gRPC and Protobuf support for remote user-defined operations

Open
#283 0 comments 0 reactions 1 assignee Claimed by @rv355 View on GitHub
Enhancement
Dominant language
C++
Stars
93
Forks
34
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**
Data movement from VDMS to remote servers for a remote UDF involves multiple filesystem read/write operations, which could result in bottlenecks, especially for data-intensive use cases. Moreover, this flow results in multiple read/write operations even at the remote server end.

**Describe the solution you'd like**
We plan to update the workflow so that instead of files, we work with protobufs, reducing the file I/O operations for both VDMS and the remote server. With protobuf as the data format, gRPC would be a better alternative to HTTP. This solution would also make writing UDFs easier, as the user would not need to worry about file read/write or the format of the files.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.