Lazy and partial message cache

Is your feature request related to a problem? Please describe.

As a user of the Mail app I notice that the first use experience is slow. That is because the app first indexes all my emails before I'm able to access a mailbox.

From a technical PoV we do this because experience has shown that IMAP search is not always reliable, especially if one wants to sort messages by their date. This feature depends on IMAP capabilities that are not always available. As a consequence, Horde falls back to a client-side pagination algorithm that fetches a full mailbox, sorts locally and then fetches the details of the calculated page. This trickled down as slow performance for our app.

Moreover ever connection to IMAP has a latency penalty for our web app as a new connection needs to be established, authentication happens, etc. A classic desktop client can leave the connection open.

Describe the solution you'd like

Relax the way the message cache works. Do not index all messages at once before we give users access to the mailbox.

Without concrete technical ideas in mind, we need to match some acceptance criteria

1. There must be an efficient way to fetch the latest x threads in a mailbox (not just messages).
2. There must be an efficient way to build message threads without having all message data available locally. We can not use IMAP threading because that is limited to a single mailbox and we want to thread across mailboxes. As in, even combine messages from Inbox and Sent so that threads appear like a conversation in a chat.
3. There must be an efficient way to access the input data we need for the importance classifier training.

Describe alternatives you've considered

N/a

Additional context

No response

No file, test, or entry point is named. Start by tracing the mailbox indexing, client-side pagination, cross-mailbox threading, and importance-classifier data paths, then define tests for each acceptance criterion: lazy latest-thread access, partial cross-mailbox threading, and efficient classifier input access.