mikel / mikel/mail

Excessive memory usage with attachments

Open
#1,602 0 comments 5 reactions 0 assignees View on GitHub
Dominant language
Ruby
Stars
3.7k
Forks
934
PR merge metrics
No merged PRs in 30d

Description

Hi all

When using the library to parse a existing message with `messsage = Mail.read("test.eml")` and then accessing the parts or attachments objects like `mesage.parts.size` or `message.attachments.size` we see excessive memory usage.

It seems like the attachments are being held fully in memory - the memory usage is very high.
For example with a message having a single attachment, originally 110 MB in size, base64-encoded in the message and thus the total size of the message coming up to 150 MB we see a overhead of memory usage of about factor two - the simple example above uses about 380 MB of memory.

Is there any option to parse a message without loading everything into memory upfront? Like in a streaming fashion?

Contributor guide

Open the contributing guide

Research direction

Start at Mail.read("test.eml") and trace what happens when parts or attachments are accessed. Compare the message size with the roughly 380 MB memory usage described, then determine whether a streaming parse can avoid loading attachment contents upfront; done means the reported excessive memory usage is addressed without breaking parts or attachments access.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
backend
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.