eclipse-ee4j / eclipse-ee4j/angus-mail

Can I force local parsing on IMAP Connections?

Open
#150 6 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Java
Stars
87
Forks
23
PR merge metrics
No merged PRs in 30d

Description

Note: This is a "repost" of https://github.com/jakartaee/mail-api/issues/733, which I created in the wrong project.

Describe the bug
We are trying to access a local-run Microsoft Exchange Server via IMAP. This leads us to retrieving attachments of incoming mails with cut-off filenames. After enabling Debug logging we can see that jakarta mail is not downloading the entire MIME structure of the e-mail but is utilizing IMAP to fetch information about attachments via the BODYSTRUCTURE call. Apparently, Exchange is too stupid to support RFC-2251 properly, which allows long fields to be split into multiple lines, and favors to return a cut-off filename instead. This was proven with the logs, the response to the BODYSTRUCTURE call contained the clipped filename (parts of the extension were missing due to them being in the next line inside the MIME structure)

Since we know that jakarta mail also supports POP3, and POP3 expects the mail to be parsed locally, is there a way to alter its behavior on IMAP connections to be the same? Our usecase mandates that we are downloading the entire mail including attachments anyways, so parsing MIME locally would both circumvent the idiocy of Exchange and not come at mentionable cost to us transmission-wise.

To Reproduce
Steps to reproduce the behavior:
Not really reproducible, this occurs on a customer system with some local Microsoft Exchange server of which operations are even outsourced to a sub-contractor...

Expected behavior
Jakarta Mail should ignore the fact that we are talking IMAP and request a download of the entire MIME structure for local parsing as it would do for POP3, instead of using IMAP commands for server-side parsing.

Screenshots
N/A

Desktop (please complete the following information):
Unknown/Irrelevant

Mail server:

Protocol being used: IMAP
Vendor/product: Microsoft Exchange
Mail service URL: company-internal

Additional context
N/A

In the other ticket, we already spoke about `mail.imaps.partialfetch=false`, however:

I tried it, but it seems it only determines if it downloads the entire bodypart in one go or in chunks. However, even with partialfetch = false, it is still handling the message parts individually by calling BODYSTRUCTURE and FETCH for individual parts of the message, as opposed to fetching the entire MIME in one go.
For comparison, when I manually connect to the Mailserver I do a
```
03 FETCH 1 BODY[]
```
which fetches the entire Mime structure, including headers, body and all attachments.
Jakartamail instead performs the following

```
C7 FETCH 1 (BODYSTRUCTURE)
* 1 FETCH (BODYSTRUCTURE (("text" "plain" ("charset" "utf-8" "format" "flowed") NIL NIL "7bit" 52 3 NIL NIL NIL NIL)("text" "plain" ("charset" "UTF-8" "name" "1.txt") NIL NIL "base64" 36 0 NIL ("attachment" ("filename" "1.txt")) NIL NIL)("text" "plain" ("charset" "UTF-8" "name" "2.txt") NIL NIL "base64" 36 0 NIL ("attachment" ("filename" "2.txt")) NIL NIL) "mixed" ("boundary" "------------379CADE6585D357FB4FBC79D") NIL ("en-US") NIL))
C7 OK Fetch completed (0.001 + 0.000 secs).
Message ID in Multipart Message is 0
C8 FETCH 1 (BODY[2])
{36}
SSBhbSB0aGUgZmlyc3QgYXR0YWNobWVudC4=)
C8 OK Fetch completed (0.001 + 0.000 secs).
C9 FETCH 1 (BODY[3])
{36}
SSBhbSB0aGUgc2Vjb25kIGF0dGFjaG1lbnQu)
C9 OK Fetch completed (0.001 + 0.000 secs).
C10 FETCH 1 (BODY[1])
{52}
Hallo,
I am an email with multiple attachments.

)
C10 OK Fetch completed (0.001 + 0.000 secs).
```
It is this way of essentially handing the MIME Parsing to the server (by asking for body parts via the protocol) that allows Exchange to botch the BODYSTRUCTURE response and cut of the file name because Exchange doesn't handle RFC-2251 properly

Contributor guide

Open the contributing guide

Research direction

The payload names no source files or tests. Start by tracing the IMAP path behind mail.imaps.partialfetch=false and compare its BODYSTRUCTURE and per-part FETCH commands with FETCH 1 BODY[]. Done means establishing whether full-message retrieval and local MIME parsing can be requested for IMAP, with coverage for the Exchange behavior described.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.