infinispan / infinispan/protostream

Reduce FileDescriptor memory usage

Open
#645 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
56
Forks
39
Avg merge
5h 36m
Merged PRs (30d)
5

Description

`FileDescriptors` keep intermediate state around, which is not needed after processing.

E.g. a single persistence.counters.proto FileDescriptor instance uses 967KB.

I'm not sure if the field documentation is useful for anything after the annotations have been parsed, but it is kept in memory, and in a hotrod-client test each DefaultCacheManager instance seem to need 3 instances of the string.
There is no native byte type in protobuf so it is mapped to int32. from the WrappedMessage.wrappedByte documentation.

Many lists are LinkedList instances, even though a LinkedList with 3 elements uses more memory than an ArrayList with 10 elements. Some lists are also wrapped in Collections$UnmodifiableRandomAccessList twice.

FileDescriptor HashMap fields (extendDescriptors, dependants) are never queried by key.

Contributor guide

Open the contributing guide

Research direction

Start by profiling a FileDescriptor instance and tracing the intermediate state, field documentation, list implementations, wrapper lists, and HashMap fields named in the issue. Determine which data remains necessary after annotation parsing and whether the maps are queried. Done means reducing retained memory without breaking descriptor processing, with relevant tests or measurements confirming the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.