apache / apache/ponymail

Bug: cluster generator fails to parse non-ascii subject and sender

Open
#513 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
JavaScript
Stars
80
Forks
32
PR merge metrics
No merged PRs in 30d

Description

The cluster generator uses fields of the msg assuming that they will be strings.

However that is not the case if non-ascii characters have been used.

In such cases, code such as msg.get('subject') will return an email.header.Header object.
This causes code such as bytes(subject, encoding = 'ascii') to fail with

TypeError: encoding without a string argument

In turn, this causes the archiver to revert to a very basic fallback mid:

mid = hashlib.sha224(str("%s-%s" % (lid, msg_metadata['archived-at'])).encode('utf-8')).hexdigest() + "@" + (lid if lid else "none")

**Unless archived-at is defined, this will be constant for a given list id**

This is relatively easy to fix; the generator should use the msg_metadata dict which
the archiver has already set up.

HOWEVER, to ensure that it's possible to regenerate the same Permalinks, any fix MUST be implemented as a new generator type, with a new syntax (i.e. change the 'r' prefix).

There are probably some other changes that need to be made to the cluster generator.
For example, Message-Id should be canonicalised.

Note that the fallback mid cannot be changed, as that would affect all the generators.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the cluster generator and the archiver's msg_metadata handling, tracing how non-ASCII subject and sender values reach the generator. Define the new generator type and changed 'r' syntax while preserving existing generator behavior and the fallback MID. Verify that regenerated permalinks remain stable and that non-ASCII messages no longer trigger the fallback.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.