Bug: cluster generator fails to parse non-ascii subject and sender
- Dominant language
- JavaScript
- Stars
- 80
- Forks
- 32
- PR merge metrics
- No merged PRs in 30d
Description
The cluster generator uses fields of the msg assuming that they will be strings.
However that is not the case if non-ascii characters have been used.
In such cases, code such as msg.get('subject') will return an email.header.Header object.
This causes code such as bytes(subject, encoding = 'ascii') to fail with
TypeError: encoding without a string argument
In turn, this causes the archiver to revert to a very basic fallback mid:
mid = hashlib.sha224(str("%s-%s" % (lid, msg_metadata['archived-at'])).encode('utf-8')).hexdigest() + "@" + (lid if lid else "none")
**Unless archived-at is defined, this will be constant for a given list id**
This is relatively easy to fix; the generator should use the msg_metadata dict which
the archiver has already set up.
HOWEVER, to ensure that it's possible to regenerate the same Permalinks, any fix MUST be implemented as a new generator type, with a new syntax (i.e. change the 'r' prefix).
There are probably some other changes that need to be made to the cluster generator.
For example, Message-Id should be canonicalised.
Note that the fallback mid cannot be changed, as that would affect all the generators.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the cluster generator and the archiver's msg_metadata handling, tracing how non-ASCII subject and sender values reach the generator. Define the new generator type and changed 'r' syntax while preserving existing generator behavior and the fallback MID. Verify that regenerated permalinks remain stable and that non-ASCII messages no longer trigger the fallback.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100