google / google/protobuf.dart

Parsing of protos from a buffer is slow when the proto / buffer is large

Open
#611 3 comments 0 reactions 0 assignees View on GitHub
perf
Dominant language
Dart
Stars
572
Forks
196
Avg merge
1h 59m
Merged PRs (30d)
2

Description

An internal 1p application uses protos in the following manner at startup. `buffer` is obtained from some sort of persistent storage.

```dart
defaultInstance.rebuild((message) => message.mergeFromBuffer(buffer));
```

A rough benchmark shows that this is slow for protos that are "large" in both the size of the buffer, and the number of fields. For example, some buffers that are 10KB in size can lead to 20ms of parsing time on a low power Android device.

See b/229065886 for more details, along with some CPU profiles.

Contributor guide

Open the contributing guide

Research direction

Start by profiling mergeFromBuffer with large buffers and many fields, using the startup pattern described in the issue and the CPU profiles referenced in b/229065886. Done means identifying and addressing the parsing bottleneck, then showing improved parsing time for comparable large-protobuf cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
dart
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.