apache / apache/couchdb

Implement quorum limit for search endpoints

Open
#4,026 2 comments 0 reactions 0 assignees View on GitHub
enhancement needs-triage
Dominant language
Erlang
Stars
7k
Forks
1.1k
Avg merge
1d 16h
Merged PRs (30d)
9

Description

## Summary

Many endpoints respond only when a quorum of replicas have responded with what the actual response should be, however not all endpoints behave this way. As @rnewson said in Slack:

> the _view, _find and _search endpoints read one copy of each shard range to form the response, there is no option to consult multiple copies, the code simply hasn't been written.

This can cause bugs in applications that make successive queries where these may be ultimately handled by different nodes.

## Desired Behaviour

All endpoints to behave consistently using the default quorum which can be overridden (eg: with `r` and `w` query parameters). This makes the behaviour of endpoints consistent and therefore easier to develop against, but also means subsequent queries can be guaranteed to work given appropriate values for the quorum.

## Possible Solution

Have these endpoints read all copies of the shard and respond when a quorum is achieved.

## Additional context

We're upgrading from supporting single node to clustered operation and certain tests in our integration suite are failing too often.

Some workarounds we have considered include, using `w=` everywhere, setting replicas to 1, and now investigating using sticky sessions so subsequent requests from the same client are handled by the same node. These have obvious drawbacks and none have been found (yet) to be 100% reliable.

Contributor guide

Open the contributing guide

Research direction

Start by tracing the _view, _find, and _search endpoint implementations and the integration tests that fail during clustered operation. Determine how replica reads and quorum parameters are handled elsewhere; done means these endpoints consult shard copies and honor the default quorum plus r and w overrides consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
erlang
Domain
api, database, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.