protocolbuffers / protocolbuffers/protobuf
[Python] Potential memory leak
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 72k
- Forks
- 16.3k
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 140
Description
What version of protobuf and what language are you using?
Version: 6.33.0
Language: Python
What operating system (Linux, Windows, ...) and version?
OS: Ubuntu Linux 22.0.1
Kernel: 6.8.0-87-generic
What runtime / compiler are you using (e.g., python version or gcc version)
Python 3.12
What did you do?
I've hacked together a little example repo (kudos to @Atheuz for the original that I've forked from), but essentially it is as follows:
obj = schema_pb2.SomeObj()
for _ in range(10_000_000):
obj.ParseFromString(b"...")
del obj
What did you expect to see
That there would be no growth in memory usage
What did you see instead?
Until the obj is deleted, the memory grew uncontrollably.
Anything else we should know about your project / environment
This is a continuation of https://github.com/protocolbuffers/protobuf/issues/10088 (I think).
I've started a Google Group Discussion where it was explained to me, that the behaviour is intended and expected.
But it is still kind of counterintuitive to see that Unpack and ParseFromString actually allocate more and more memory.
I think it deserves at least a mention in the docs on how to properly use them. And thus I was advised to open an issue here.
Thank you!
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the issue's ParseFromString and Unpack examples, then review the existing Python protobuf documentation for the appropriate location. Confirm the intended memory behavior in the linked Google Group discussion and explain how users should interpret or avoid apparent growth; done means the relevant documentation clearly covers this behavior and usage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100