Slow 'startswith'-queries due to '$options': 'u' being set (no index usage?)

On Mongoengine 0.20.0 and mongodb 4.4.0

Im doing a fairly involved query setting up a couple of Qs (and/or-ing) , some of them using "startswith" operation that translates into the $regex with ^-anchor. According to doc we should be able to use the indexes in that case:

https://docs.mongodb.com/manual/reference/operator/query/regex/#mongodb-query-op.-regex

(Ie. q = Q(unit_fqn__startswith=fqn))

It was very slow fetching counts or results.

Using the explain() method I figured out the query built by mongoengine and tried executing it separately using the mongo shell. I found out that it was the '$options': 'u' (that is re.UNICODE) causing the slow query on the regex search.

EXAMPLE:

First query is very slow (many seconds), index not used (according to explain()), second is close to instant (collection size 750K documents):

1. (SLOW)
   db.employee.find( {'$and': [{'$or': [{'unit_fqn': {'$regex': '^Comapny\ Global\ \(TEST\ KP\)˛Portugal˛', '$options': 'u'}}, {'unit_fqn': {'$regex': '^Comapny\ Global\ \(TEST\ KP\)˛Filippinerna˛'}}]}, {'hidden': {'$eq': false}}]})

2. (FAST)
   db.employee.find( {'$and': [{'$or': [{'unit_fqn': {'$regex': '^Comapny\ Global\ \(TEST\ KP\)˛Portugal˛'}}, {'unit_fqn': {'$regex': '^Comapny\ Global\ \(TEST\ KP\)˛Filippinerna˛'}}]}, {'hidden': {'$eq': false}}]})

Not saying this is a bug, but this was unexpected behaviour as Im not doing anything out of the ordinary. I have not yet found out what is causing the option to be set (see comment below though). What am I missing?

Start with the Q(unit_fqn__startswith=fqn) query translation in MongoEngine 0.20.0 and inspect the generated MongoDB regex options. Reproduce the two queries against MongoDB 4.4.0, compare explain() output, and determine whether the unexpected Unicode option is intended. Done means the cause and expected behavior are established, with a focused fix or documentation change identified.