matrix-org / matrix-org/matrix-spec

Flatbuffer binary encoding of existing json spec?

Open
#236 2 comments 0 reactions 0 assignees View on GitHub
A-Client-Server feature
Dominant language
HTML
Stars
330
Forks
150
Avg merge
2h 21m
Merged PRs (30d)
3

Description

Would a flatbuffer binary encoding of the existing json protocol spec for the matrix events be considered?

I experimented with encoding the existing json schema as binary using flatbuffers, and it did appear to work alright for `m.room.member_example.json`. I presume it would be fine for other messages, and that the flatbuffers schemas could be automatically generated with a tool from the matrix-spec repo (Meaning potentially no need to maintain a separate binary protocol specification).

I published my test here - https://github.com/matrix-hacks/matrix-flatbuffers

... I believe this would allow clients and servers to completely ditch the need for json parsing/encoding which relieves clients of a lot of their code size and complexity, while increasing message handling performance significantly -- all relevant for minimal/embedded clients written in C and other low level languages.

Flatbuffers already has language bindings for Go (and python and C++ and javascript and many others).

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing JSON protocol specification in matrix-spec and the proof of concept in matrix-hacks/matrix-flatbuffers. Determine whether schemas can be generated for the other Matrix events and whether the proposed encoding preserves protocol compatibility. Done would require an agreed design and implementation scope for binary client and server support.

Written by the indexing model from the issue text.

Assessment

Tech stack
json
Domain
api, backend-api-design
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.