apache / apache/ponymail

Archiving emails in violation of RFC-2821 line-endings may result in multiple emails on a redundant setup

Open
#394 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
80
Forks
32
PR merge metrics
No merged PRs in 30d

Description

It would seem that when some (older) MTAs send out email, they do not conform to RFC-2821 about newlines. From the RFC, it is stated that:
~~~
In addition, the appearance of "bare" "CR" or "LF" characters in text
(i.e., either without the other) has a long history of causing
problems in mail implementations and applications that use the mail
system as a tool. SMTP client implementations MUST NOT transmit
these characters except when they are intended as line terminators
and then MUST, as indicated above, transmit them only as a
sequence.
~~~

Case in point: qmail sometimes will send an email using only LF instead of CRLF. This is then corrected to CRLF by postfix, but has the disadvantage in clustered setups that one archiver may receive the original input while the next gets the corrected one. The difference there is but a single added newline character, but that is enough to cause two distinct IDs being generated.

Short of fixing all MTAs, the best solution seems to be detecting any STDIN that ends in a double newline and, if found, crop the last one out before archiving.

The fix seems to be as simple as (in archiver.py, line 580-ish):

~~~
if msgstring[-2:] == b'\n\n':
msgstring = msgstring[:-1]
~~~

I'll investigate further and implement a solution when I am satisfied this will resolve the issue.

Contributor guide

No contributing guide indexed for this repository

Research direction

Read archiver.py around line 580 and exercise its STDIN path with messages ending in double newlines, including qmail-style LF input and CRLF input. Done means equivalent messages no longer produce distinct archive IDs in a redundant setup, while normal message endings remain unchanged.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.