apache / apache/lucene

CPU usage continuously increasing when IMAP SEARCH command coming from user agent

Open
#12,209 1 comment 0 reactions 0 assignees View on GitHub
type:bug
Dominant language
Java
Stars
3.6k
Forks
1.4k
Avg merge
2d 11h
Merged PRs (30d)
88

Description

### Description

We use Apache James to provide voice visual mail for iPhone users and Lucene module is used in James to store nessecary information for index search.

But the CPU usage of James server is continuously increasing like this:
![MicrosoftTeams-image](https://user-images.githubusercontent.com/128135965/225887687-05f06f26-d4fd-4c9f-bf7d-11e25f6c22ee.png)

Below is the command sequence from iPhone:
- SELECT INBOX
- UID SEARCH 1:* NOT DELETED
- UID SEARCH 1:* UNSEEN
- UID STORE [uid_1] +FLAGS.SILENT (\Seen))
- UID STORE [uid_2] +FLAGS.SILENT (\Seen))
- ......
- LOGOUT

After several tests, we think maybe there are some problems in the implement of SEARCH command, so we change the search query command request to
- UID SEARCH UID ALL

instead of 1:* NOT/UN~ in our test environment.
Then the CPU usage became stable like this:
![MicrosoftTeams-image (3)](https://user-images.githubusercontent.com/128135965/225890304-e0c5a474-3342-487f-ad39-8b8710ca961d.png)

We checked sourcecode and found there is a block in James like below:

`public Flux search(SearchQuery query, MailboxSession mailboxSession) throws MailboxException {`
` if (query.equals(LIST_ALL_QUERY) || query.equals(LIST_FROM_ONE)) {`
` return listAllMessageUids(mailboxSession);`
` }`
` return index.search(mailboxSession, getMailboxEntity(), query);`
`}`

When the query type is LIST_ALL_QUERY or LIST_FROM_ONE, it will go into the if block, not using index search of Lucene and the CPU usage is stable. Otherwise, in the other query cases when index.search method being called, the CPU usage will increase.

The question is why. Is there some performance problem in Lucene, or this issue can be resolved by correct configuration?

### Version and environment details

James: 3.6.0
Lucene: 3.6.2
JVM: OpenJDK 64-Bit Server VM
OS: CentOS7.2

Contributor guide

Open the contributing guide

Research direction

Start with the mailbox search method containing the LIST_ALL_QUERY and LIST_FROM_ONE branch, then trace the index.search call used for other queries. Reproduce the reported UID SEARCH patterns on James 3.6.0 with Lucene 3.6.2 and compare CPU behavior; done means identifying whether the increase is caused by search performance or configuration and documenting a verified resolution.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.