Cannot use Token.ZeroWidthEscape with pygments

I had a patch at https://github.com/jonathanslenders/python-prompt-toolkit/pull/615 that fixed this, but it was never merged. Right now I have to use

class MyPygmentsTokens(PygmentsTokens):
    """
    Support ZeroWidthEscape
    """

    def __pt_formatted_text__(self):
        result = []

        for token, text in self.token_list:
            if list(token) == ['ZeroWidthEscape']:
                result.append(('[ZeroWidthEscape]', text))
            else:
                result.append(('class:' + pygments_token_to_classname(token), text))

        return result

and use MyPygmentsTokens everywhere instead of PygmentsTokens.

Questions I never got answers to:

• Why can't https://github.com/jonathanslenders/python-prompt-toolkit/pull/615/commits/33d47a1f25ec4b10dacb7fb83f7bb91512d0e2c5 be merged?

• If it can't be included, how do I use zero width escape sequences alongside pygments theming? Is subclassing PygmentsTokens my only recourse?

Start by reviewing pull request 615 and commit 33d47a1f25ec4b10dacb7fb83f7bb91512d0e2c5, then trace how PygmentsTokens handles Token.ZeroWidthEscape. Determine why the proposed change was not merged and verify that zero-width escape sequences work alongside Pygments theming without requiring a subclass.