Tuple formats in IPROTO

Product: Tarantool
Since: 3.0
Root document:
https://www.tarantool.io/en/doc/latest/dev_guide/internals/msgpack_extensions/
https://www.tarantool.io/en/doc/latest/dev_guide/internals/box_protocol/
https://www.tarantool.io/en/doc/latest/reference/reference_lua/compat/

SME: @ CuriousGeorgiy

Details

New MsgPack extension was added, MP_TUPLE, which is used exclusively
in IPROTO. It has the following structure:
+----------+-----------------+----------+
|MP_EXT = 7| MP_UINT | MP_ARRAY |
+----------+-----------------+----------+
^ ^
format identifier tuple data

New IPROTO features were added:

• IPROTO_FEATURE_DML_TUPLE_EXTENSION = 7 — tuples in IPROTO_DATA are
  encoded as MP_TUPLE and tuple format is sent in response to DML
  requests.
• IPROTO_FEATURE_CALL_RET_TUPLE_EXTENSION = 8 —tuples in IPROTO_DATA
  are encoded as MP_TUPLE and tuple format is sent in response to call
  and eval requests.
• IPROTO_FEATURE_CALL_ARG_TUPLE_EXTENSION = 9 — tuples in IPROTO_TUPLE
  of call and eval requests are encoded as MP_TUPLE with their
  formats encoded as described below.

The IPROTO_FEATURE_DML_TUPLE_EXTENSION feature can be enabled in
net.box by setting the fetch_schema option to false when creating a
new connection.

A new IPROTO key for bodies of responses with data and bodies of call and
eval requests: IPROTO_TUPLE_FORMATS = 0x60 — a MP_MAP consisting of
MP_UINT keys (format identifiers) and MP_STRING values (tuple format
clauses).

A new backward compatibility option was added (and a new page,
https://tarantool.io/compat/box_tuple_extension, needs to be created for
it): BOX_TUPLE_EXTENSION_BRIEF — it controls the
IPROTO_FEATURE_CALL_RET_TUPLE_EXTENSION and
IPROTO_FEATURE_CALL_ARG_TUPLE_EXTENSION feature bits. It was added for
clients that assume to receive MP_ARRAYs in call and eval requests
arguments and response data.

Please refer to the design document for details.
Requested by @CuriousGeorgiy in https://github.com/tarantool/tarantool/commit/d79e7a3c05fb6d8d638e989093d82293f2f894b2.

Document:

• MP_TUPLE MsgPack extension
• IPROTO features
• IPROTO keys
• box_tuple_extension compat option

Start with the three root documents listed in the issue: MsgPack extensions, the IPROTO protocol, and Lua compatibility. Use the linked design document to verify the MP_TUPLE structure, feature bits, IPROTO_TUPLE_FORMATS, and BOX_TUPLE_EXTENSION_BRIEF details. Done means documenting all four checklist items, including creating the box_tuple_extension compatibility page.