Broken completion with diacritics

Working

Take the following code

import prompt_toolkit.completion
comp = prompt_toolkit.completion.WordCompleter(["foo", "bar"])
prompt_toolkit.prompt(completer=comp)

Test it

• press f
• prompt_toolkit suggests foo -> good
• press o
• prompt_toolkit still suggests foo -> good

Not working

Take the following code

import prompt_toolkit.completion
comp = prompt_toolkit.completion.WordCompleter(["aézzz", "éxxx", "bar", "xfail"])
prompt_toolkit.prompt(completer=comp)

Test it

• press a
• prompt_toolkit suggests aézzz -> good
• press é
• prompt_toolkit suggests éxxx -> wrong
• press z
• prompt_toolkit does not suggest anything -> wrong
• tab does not complete -> wrong

Try again

• press a
• prompt_toolkit suggests aézzz -> good
• press é
• prompt_toolkit suggests éxxx -> wrong
• press x
• prompt_toolkit suggests xfail -> wrong

Start by running the two prompt_toolkit completion snippets and compare the behavior after each listed keystroke. The fix is done when completion keeps the expected diacritic-containing word selected and typing further characters or pressing Tab completes it correctly.