apache / apache/couchdb

Long running erlang map/reduce can block view compaction from completion, leaking erlang procs

Open
#4,725 7 comments 0 reactions 0 assignees View on GitHub
bug needs-triage
Dominant language
Erlang
Stars
7k
Forks
1.1k
Avg merge
1d 16h
Merged PRs (30d)
9

Description

## Description

A long running/slow erlang map/reduce due to a new shard deployment, appears to be blocking that shards view compaction from completing. It also appears to be leaking/growing erlang procs at a steady rate, between 5k-10k per hour.

## Steps to Reproduce

Start view Compaction
Start long Erlang/reduce
View Compaction tries to complete, is unable to until indexer completes (suspected, waiting to observe this outcome)
Observe steady increase in erlang procs (may require continued insertion/interaction with the shard)

## Expected Behaviour

View compaction should not be blocked
Erlang procs should not continue to increase until it hits the limit and crashes

## Your Environment
AWS C6i.x32large 5 nodes q=3 n=5

* CouchDB version used: 3.2.2
* Operating system and version: Debian Buster

## Additional Context
We resharded which resulted in the erlang map reduce being a lot longer than it should(not incremental).

Contributor guide

Open the contributing guide

Research direction

No source files or tests are named. Start by reproducing the view-compaction and long Erlang map/reduce sequence after a shard deployment, while monitoring compaction completion and Erlang process counts. Done means compaction is not blocked and the process count does not steadily grow toward the limit.

Written by the indexing model from the issue text.

Assessment

Tech stack
erlang
Domain
databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.