protocolbuffers / protocolbuffers/protobuf-javascript

Javascript code is extraordinarily slow

Open
#39 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

javascript P3 triaged
Dominant language
JavaScript
Stars
471
Forks
91
Avg merge
3h 57m
Merged PRs (30d)
2

Description

Version: 3.9.1
Language: Javascript

I have a message like this:

message Outer {
  repeated Inner inner = 1;
}
message Inner {
  repeated Item items = 1;
}
message Item {
  uint32 a = 1;
  uint32 b = 2;
  uint32 c = 3;
  uint32 d = 4;
  uint32 e = 5;
}

It contains approximately 1000 Inners and 700k Items in total and comes out at about 10MB. Using the default Javascript implementation of protobufs it takes 10 seconds to decode. I switched to using Mapbox pbf, and that only takes 170 milliseconds.

I looked at PBF's code and they don't seem to be doing anything special. decodeVarint looks like it has had some work to make Javascript engines happy with varints normally being less than 2^32, but... come on! That's 2 orders of magnitude! How is this implementation so slow?

Another issue is that this code generates objects with getters and setters, rather than just creating a plain object. Why? It just means I have to do even more tedious work when my message has been decoded to turn it into a form that I can easily use!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by profiling the default JavaScript protobuf decoder against the reported nested schema and approximately 10MB dataset, then compare its varint decoding and object construction with Mapbox PBF. Determine whether the performance gap and getter/setter behavior represent actionable changes; done means a scoped fix with measurements and tests showing the improvement.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
backend
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.