protocolbuffers / protocolbuffers/protobuf

[Python] Potential memory leak

Open Beginner friendly
#24,257 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

help wanted python
Dominant language
C++
Stars
72k
Forks
16.3k
Avg merge
1d 17h
Merged PRs (30d)
140

Description

What version of protobuf and what language are you using?
Version: 6.33.0
Language: Python

What operating system (Linux, Windows, ...) and version?
OS: Ubuntu Linux 22.0.1
Kernel: 6.8.0-87-generic

What runtime / compiler are you using (e.g., python version or gcc version)
Python 3.12

What did you do?
I've hacked together a little example repo (kudos to @Atheuz for the original that I've forked from), but essentially it is as follows:

obj = schema_pb2.SomeObj()

for _ in range(10_000_000):
    obj.ParseFromString(b"...")

del obj

What did you expect to see

That there would be no growth in memory usage

What did you see instead?

Until the obj is deleted, the memory grew uncontrollably.

Anything else we should know about your project / environment

This is a continuation of https://github.com/protocolbuffers/protobuf/issues/10088 (I think).

I've started a Google Group Discussion where it was explained to me, that the behaviour is intended and expected.

But it is still kind of counterintuitive to see that Unpack and ParseFromString actually allocate more and more memory.
I think it deserves at least a mention in the docs on how to properly use them. And thus I was advised to open an issue here.

Thank you!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the issue's ParseFromString and Unpack examples, then review the existing Python protobuf documentation for the appropriate location. Confirm the intended memory behavior in the linked Google Group discussion and explain how users should interpret or avoid apparent growth; done means the relevant documentation clearly covers this behavior and usage.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.