apache / apache/couchdb

[Bug]: selector-filtered _changes is slower the first time after creating a Mango index

Open
#5,926 1 comment 0 reactions 0 assignees View on GitHub
bug needs-triage
Dominant language
Erlang
Stars
7k
Forks
1.1k
Avg merge
1d 16h
Merged PRs (30d)
9

Description

### Version

3.5.1, and `main` branch

### Describe the problem you're encountering

Despite the fact that `_changes` does not use indexes to speed up selector filters, the first time it is requested after creating a Mango index, it is slower, which possibly indicates it's spending time updating the index.

### Expected Behaviour

The time taken for `_changes` to respond should not depend on the existence or otherwise of Mango indexes.

### Steps to Reproduce

To demonstrate this, I created a DB containing a million docs of the form `{ "_id": "doc-N", "value": N }` for N from 1 to 1,000,000. Then I timed how long it took to run this filtered changes request:

```sh
# uses this shell function:
cdb () {
curl -gsu 'admin:admin' "http://127.0.0.1:5984$1" "${@:2}"
}

time ( cdb '/db/_changes?filter=_selector' -X POST -d '{ "selector": { "value": { "$lt": 100 } } }' )
```

This would take around 12 seconds.

Then I created an index covering the filtered field and re-timed:

```sh
cdb '/db/_index' -X POST -d '{ "index": { "fields": ["value"] }, "name": "by-value", "type": "json" }'
time ( cdb '/db/_changes?filter=_selector' -X POST -d '{ "selector": { "value": { "$lt": 100 } } }' )
```

The first time after creating an index, `_changes` takes over 30s to complete. On later requests it's back to around 12s, possibly very slightly slower.

This effect shows up regardless of which fields the index covers; even if it does not cover the `_changes` selector, the `_changes` request is still slow.

### Your Environment

macOS 15, CouchDB 3.5.1, single node, database created with q=2, n=1

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Research direction

Reproduce the slowdown with the provided curl commands using a large database, then compare the first and later selector-filtered _changes requests before and after creating the _index entry. Trace the _changes selector-filtering path alongside index creation or update handling. Done means the first filtered _changes request is not delayed by unrelated Mango indexes, with regression coverage for the reproduced case.

Written by the indexing model from the issue text.

Assessment

Tech stack
erlang
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.