GAM-team / GAM-team/got-your-back

Unhandled exception when cleaning message with unicode/emoji in (From:) headers.

Open
#433 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
3.1k
Forks
245
PR merge metrics
No merged PRs in 30d

Description

**Full steps to reproduce the issue:**
1. Backup email with message that ~is not saved in UTF8 format~ has unicode/emoji in `From:` header.
2. Restore email using `--cleanup`.

**Expected outcome:** GYB gracefully handles unicode/emoji in headers, either:
- Detecting/reading non UTF8 messages with appropriate encoding.
- Skipping message.

**Actual outcome:** GYB exits with unhandled exception:
```
Traceback (most recent call last):166783)
File "gyb.py", line 2767, in
File "gyb.py", line 2239, in main
File "gyb.py", line 1947, in message_hygiene
File "gyb.py", line 1891, in cleanup_from
File "email\utils.py", line 215, in parseaddr
File "email\_parseaddr.py", line 517, in __init__
File "email\_parseaddr.py", line 260, in getaddrlist
TypeError: object of type 'Header' has no len()
[31420] Failed to execute script 'gyb' due to unhandled exception!
```

**Work-around:**
- ~Convert offending .eml file to UTF8 format.~ Doesn't always work...
- Rename .eml file so GYB skips this message.

**Suggested alternative fix:** always convert non UTF8 files to UTF8 when saving backup.

**Notes:**
- The offending email is restored without error if `--cleanup` is not used. (Did not confirm if text was mangled after restore.)
- The .eml file was generated by `gyb --action backup`.
- Vim tries to open the file with latin1 encoding, but the text is mangled.
- Notepad.exe tries to open the file with UTF8 encoding, but the text is mangled.
- The Gmail 'Download Original' file does *not* work: text is still mangled.
- Instead, the Gmail 'View Original' text had to be manually copied and saved as a text file (with encoding UTF-16 LE).
- (Creating a blank UTF8 file and pasting Gmail 'View Original' text seems to work, too)
- The problematic text is in Korean.
- I was able to create a minimal repro of this issue in the python REPL:

```
Python 3.11.4 (tags/v3.11.4:d2340ef, Jun 7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import email
>>> f = open('2021/8/14/17b437668e8b5c17.eml', 'rb')
>>> bytes = f.read()
>>> m = email.message_from_bytes(bytes)
>>> m['to']
'J***********y'
>>> m['from']

>>> len(m['from'])
Traceback (most recent call last):
File "", line 1, in
TypeError: object of type 'Header' has no len()
```

Mangled text:
```
From: "(주)한웰이쇼핑"
To: J***********y
Subject: [´ÙÀ̼Ҹô] °³ÀÎÁ¤º¸ À¯È¿±â°£Á¦¿¡ µû¸¥ ÈÞ¸é°èÁ¤ Àüȯ ¾È³»µå¸³´Ï´Ù.
```

Proper text:
```
From: "(주)한웰이쇼핑"
To: "J***********y"
Subject: [다이소몰] 개인정보 유효기간제에 따른 휴면계정 전환 안내드립니다.
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with gyb.py's message_hygiene and cleanup_from path, then reproduce the failure by restoring the problematic .eml with --cleanup. Inspect the reported email.utils.parseaddr failure and verify that cleanup handles the Unicode or non-UTF8 From: header without an unhandled exception, either by reading it appropriately or skipping the message.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.