tarantool / tarantool/doc

Protobuf module

Open Beginner friendly
#4,379 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

3.2 reference
Dominant language
CSS
Stars
15
Forks
49
Avg merge
1d 13h
Merged PRs (30d)
3

Description

Related dev. issue(s): https://github.com/tarantool/tarantool/issues/9844

Product: Tarantool
Since: 3.2
Root document: a new page in https://www.tarantool.io/en/doc/latest/reference/reference_lua/
SME: @ Col-Waltz

Details

Introducing protobuf encoder. All the Protocol Buffers wire types
are supported except group start/end, which are deprecated in proto3.
To encode data you need to create a protocol according to which
data will be encoded.

The two main components of the protocol are messages and enums.
To create them .message and .enum functions are used

Each message consists of name of the message and fields.
Each field has a name, a type and an id. Id must be unique
for all fields in one message. For example this is a message with
six fields:

protobuf.message('KeyValue', {
    key = {'bytes', 1},
    create_revision = {'int64', 2},
    mod_revision = {'int64', 3},
    version = {'int64', 4},
    value = {'bytes', 5},
    lease = {'int64', 6),
})

This implementation supports recursive definition for fields in
message. Depth of recursion is defined by input data because all
fields are optional by default. Example of recursive message:

protobuf.message('Node', {
    number = {'int64', 1},
    data = {'bytes', 2},
    next_node = {'Node', 3},
})

Each enum type consists of name of type and values.
Values must have a zero value to be set as default as in example:

protobuf.enum('EventType', {
    ['PUT'] = 0,
    ['DELETE'] = 1,
})

To create a protocol .protocol function is used. This function
supports forward declared types and nested messages so the tuple
can be set according to example:

schema = protobuf.protocol({
    protobuf.message(<...>),
    protobuf.message(<...>),
    protobuf.enum(<...>),
    protobuf.enum(<...>),
})

Output protocol can then be used for encoding entered data by the
method named encode. This method converts input data
according to chosen message definition from
protobuf protocol into protobuf wireformat. All fields in message
definition are optional so if some input data is missing
it simply will not be encoded. Input data can be
submitted using luatypes or using cdata (for example
entering int64) according to the example.

result = schema:encode(‘KeyValue’,
    {
        key = 'protocol',
        version = 2,
        lease = 5,
    }
)

Output result will be a binary string encoded according to the
protobuf standard and can be transmitted to another user.
Requested by @ Col-Waltz in https://github.com/tarantool/tarantool/commit/f83fded700e89869a31bb282c1007dfd30560330.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with related development issue #9844 and the reference_lua section identified as the root document. Add a new page covering protobuf encoding, message and enum definitions, protocols, recursive fields, supported wire types, optional input fields, luatypes, cdata, and the provided examples; done means the encoder behavior is documented for Tarantool users.

Written by the indexing model from the issue text.

Assessment

Tech stack
lua
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.