tarantool / tarantool/tarantool

Support GRPC as transport

Open
#5,764 1 comment 6 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature in design Initiative
Dominant language
Lua
Stars
3.7k
Forks
419
Avg merge
1d 23h
Merged PRs (30d)
88

Description

Create a module (built-in? external?) that will provide ability to define an GRPC protocol/endpoint that will be served using user defined Lua function(s). I see the following subtasks:

  1. Parse and compile a GRPC protocol/endpoint description, which is given as a Lua string (at least assume the compilation step in the API to have ability to implement it later).
  2. Parse a GRPC request and validate it against given GRPC protocol/endpoint description. The result of parsing / validation should be either a meaningful error or a request in the Lua format.
  3. Provide ability to encode a GRPC response from a Lua value with validation against given protocol/endpoint definition.
  4. Consider providing a helper for using the module together with the tarantool/httpng module: a way to acquire a handler for the http server for a particular protocol/endpoint that will be served by user defined Lua function(s).

Many details are to be investigated and defined better. That is just my very high level vision.

BTW, we can implement a protobuf schema / binary format support as a separate module. Also maybe it is good to start from just protobuf, because GRPC is based on it (AFAIK).


The original wording describes various tasks around GRPC: expose iproto via GRPC, expose a user defined endpoint, connect to an external service. We decided to shrink the scope of this task and concentrate of the second point. See the original description below (it can be used as base for future issues).

The original description

There are several major goals:

  1. Use GRPC as a transport to access spaces and functions exposed via our native binary protocol (iproto). Popular programming languages usualy have GRPC libraries and so it should simplify development of connectors.
  2. Define an application / a module specific GRPC endpoints.
  3. Connect to external services from tarantool using GRPC.

In my understanding, the second and third points assumes that we should provide ability to complile a protocol definition in runtime, otherwise it will not convenient to use from a Lua application / module.

Only the first point is the definition of done of the issue, however the overall design should foresee reusage of significant code parts.

We plan to reuse our future H2O based HTTP/2 server.

@Mons said that he has a vision how to better implement GRPC support for Lua.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. Start by reading the narrowed scope in the issue and clarifying the GRPC endpoint, protocol-description, request-validation, and response-encoding requirements. Done means a defined implementation plan and a user-defined Lua-served GRPC endpoint with the required validation and encoding behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
lua
Domain
api, backend-api-design, networking
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.