GAM-team / GAM-team/got-your-back
Unhandled exception when cleaning message with unicode/emoji in (From:) headers.
- Dominant language
- Python
- Stars
- 3.1k
- Forks
- 245
- PR merge metrics
- No merged PRs in 30d
Description
**Full steps to reproduce the issue:**
1. Backup email with message that ~is not saved in UTF8 format~ has unicode/emoji in `From:` header.
2. Restore email using `--cleanup`.
**Expected outcome:** GYB gracefully handles unicode/emoji in headers, either:
- Detecting/reading non UTF8 messages with appropriate encoding.
- Skipping message.
**Actual outcome:** GYB exits with unhandled exception:
```
Traceback (most recent call last):166783)
File "gyb.py", line 2767, in
File "gyb.py", line 2239, in main
File "gyb.py", line 1947, in message_hygiene
File "gyb.py", line 1891, in cleanup_from
File "email\utils.py", line 215, in parseaddr
File "email\_parseaddr.py", line 517, in __init__
File "email\_parseaddr.py", line 260, in getaddrlist
TypeError: object of type 'Header' has no len()
[31420] Failed to execute script 'gyb' due to unhandled exception!
```
**Work-around:**
- ~Convert offending .eml file to UTF8 format.~ Doesn't always work...
- Rename .eml file so GYB skips this message.
**Suggested alternative fix:** always convert non UTF8 files to UTF8 when saving backup.
**Notes:**
- The offending email is restored without error if `--cleanup` is not used. (Did not confirm if text was mangled after restore.)
- The .eml file was generated by `gyb --action backup`.
- Vim tries to open the file with latin1 encoding, but the text is mangled.
- Notepad.exe tries to open the file with UTF8 encoding, but the text is mangled.
- The Gmail 'Download Original' file does *not* work: text is still mangled.
- Instead, the Gmail 'View Original' text had to be manually copied and saved as a text file (with encoding UTF-16 LE).
- (Creating a blank UTF8 file and pasting Gmail 'View Original' text seems to work, too)
- The problematic text is in Korean.
- I was able to create a minimal repro of this issue in the python REPL:
```
Python 3.11.4 (tags/v3.11.4:d2340ef, Jun 7 2023, 05:45:37) [MSC v.1934 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import email
>>> f = open('2021/8/14/17b437668e8b5c17.eml', 'rb')
>>> bytes = f.read()
>>> m = email.message_from_bytes(bytes)
>>> m['to']
'J***********y'
>>> m['from']
>>> len(m['from'])
Traceback (most recent call last):
File "", line 1, in
TypeError: object of type 'Header' has no len()
```
Mangled text:
```
From: "(주)한웰ì´ì‡¼í•‘"
To: J***********y
Subject: [´ÙÀ̼Ҹô] °³ÀÎÁ¤º¸ À¯È¿±â°£Á¦¿¡ µû¸¥ ÈÞ¸é°èÁ¤ Àüȯ ¾È³»µå¸³´Ï´Ù.
```
Proper text:
```
From: "(주)한웰이쇼핑"
To: "J***********y"
Subject: [다이소몰] 개인정보 유효기간제에 따른 휴면계정 전환 안내드립니다.
```
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with gyb.py's message_hygiene and cleanup_from path, then reproduce the failure by restoring the problematic .eml with --cleanup. Inspect the reported email.utils.parseaddr failure and verify that cleanup handles the Unicode or non-UTF8 From: header without an unhandled exception, either by reading it appropriately or skipping the message.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100