Long running erlang map/reduce can block view compaction from completion, leaking erlang procs
- Dominant language
- Erlang
- Stars
- 7k
- Forks
- 1.1k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 9
Description
## Description
A long running/slow erlang map/reduce due to a new shard deployment, appears to be blocking that shards view compaction from completing. It also appears to be leaking/growing erlang procs at a steady rate, between 5k-10k per hour.
## Steps to Reproduce
Start view Compaction
Start long Erlang/reduce
View Compaction tries to complete, is unable to until indexer completes (suspected, waiting to observe this outcome)
Observe steady increase in erlang procs (may require continued insertion/interaction with the shard)
## Expected Behaviour
View compaction should not be blocked
Erlang procs should not continue to increase until it hits the limit and crashes
## Your Environment
AWS C6i.x32large 5 nodes q=3 n=5
* CouchDB version used: 3.2.2
* Operating system and version: Debian Buster
## Additional Context
We resharded which resulted in the erlang map reduce being a lot longer than it should(not incremental).
Contributor guide
Research direction
No source files or tests are named. Start by reproducing the view-compaction and long Erlang map/reduce sequence after a shard deployment, while monitoring compaction completion and Erlang process counts. Done means compaction is not blocked and the process count does not steadily grow toward the limit.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- erlang
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100