Excessive memory usage with attachments
- Dominant language
- Ruby
- Stars
- 3.7k
- Forks
- 934
- PR merge metrics
- No merged PRs in 30d
Description
Hi all
When using the library to parse a existing message with `messsage = Mail.read("test.eml")` and then accessing the parts or attachments objects like `mesage.parts.size` or `message.attachments.size` we see excessive memory usage.
It seems like the attachments are being held fully in memory - the memory usage is very high.
For example with a message having a single attachment, originally 110 MB in size, base64-encoded in the message and thus the total size of the message coming up to 150 MB we see a overhead of memory usage of about factor two - the simple example above uses about 380 MB of memory.
Is there any option to parse a message without loading everything into memory upfront? Like in a streaming fashion?
Contributor guide
Research direction
Start at Mail.read("test.eml") and trace what happens when parts or attachments are accessed. Compare the message size with the roughly 380 MB memory usage described, then determine whether a streaming parse can avoid loading attachment contents upfront; done means the reported excessive memory usage is addressed without breaking parts or attachments access.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ruby
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100