apache / apache/couchdb

CouchDB compression error

Open
#4,035 7 comments 0 reactions 0 assignees View on GitHub
bug needs-triage
Dominant language
Erlang
Stars
7k
Forks
1.1k
Avg merge
1d 16h
Merged PRs (30d)
9

Description

Recently we are noticing couchdb error frequently in one of our database
![image](https://user-images.githubusercontent.com/841011/169829614-e4e68f4b-e149-4e1e-9a17-35665241f057.png) in RedHat Linux server.

We are using CouchDB version 3.1.1, installed through Docker Container.

Issue is happening only in sensor_event database which has about 280 GB of data.

Running in one node.

Currently we are rolling back the entire VM of the system to previous working CouchDB day. This causes loss of data to our study. Is there a solution to having this issue. Thanks!

Here are the settings we have

**Section Option Value**
attachments compressible_types text/*, application/javascript, application/json, application/xml
compression_level 8
chttpd backlog 512
bind_address 0.0.0.0
max_db_number_for_dbs_info_req 100
port 5984
prefer_minimal Cache-Control, Content-Length, Content-Range, Content-Type, ETag, Server, Transfer-Encoding, Vary
require_valid_user false
server_options [{recbuf, undefined}]
socket_options [{sndbuf, 262144}, {nodelay, true}]
cluster n 3
q 2
compactions _default [{db_fragmentation, "70%"}, {view_fragmentation, "60%"}, {from, "02:00"}, {to, "05:00"}]
cors credentials false
couch_httpd_auth allow_persistent_cookies true
auth_cache_size 50
authentication_db _users
authentication_redirect /_utils/session.html
iterations 10
require_valid_user false
secret fc7857a6df7358f9ccefc842ecb9e0e2
timeout 600
couch_peruser database_prefix userdb-
delete_dbs false
enable false
couchdb attachment_stream_buffer_size 4096
changes_doc_ids_optimization_threshold 100
database_dir ./data
default_engine couch
default_security admin_only
file_compression none
max_dbs_open 500
max_document_size 8000000
os_process_timeout 5000
users_db_security_editable false
uuid 4e5744d50ad4d7d3d3a189b474f850c7
view_index_dir ./data
couchdb_engines couch couch_bt_engine
csp enable true
feature_flags partitioned||* true
httpd allow_jsonp false
authentication_handlers {couch_httpd_auth, cookie_authentication_handler}, {couch_httpd_auth, default_authentication_handler}
bind_address 127.0.0.1
enable_cors false
enable_xframe_options false
max_http_request_size 4294967296
port 5986
secure_rewrites true
socket_options [{sndbuf, 262144}]
indexers couch_mrview true
ioq concurrency 10
ratio 0.01
ioq.bypass compaction false
os_process true
read true
shard_sync false
view_update true
write true
log level info
writer stderr
query_server_config os_process_limit 100
reduce_limit true
replicator connection_timeout 30000
http_connections 20
interval 60000
max_churn 20
max_jobs 500
retries_per_request 5
socket_options [{keepalive, true}, {nodelay, false}]
ssl_certificate_max_depth 3
startup_jitter 5000
verify_ssl_certificates false
worker_batch_size 500
worker_processes 4
ssl port 6984
uuids algorithm sequential
max_count 1000
vendor name The Apache Software Foundation

**Here is the error**
```
Error running query. Reason: (invalid_compression) [{couch_compress,decompress,1,[{file,"src/couch_compress.erl"},{line,71}]}, {couch_file,pread_term,2,[{file,"src/couch_file.erl"},{line,161}]}, {couch_btree,get_node,2,[{file,"src/couch_btree.erl"},{line,435}]}, {couch_btree,modify_node,4,[{file,"src/couch_btree.erl"},{line,388}]}, {couch_btree,modify_kpnode,6,[{file,"src/couch_btree.erl"},{line,535}]}, {couch_btree,modify_node,4,[{file,"src/couch_btree.erl"},{line,394}]}, {couch_btree,modify_kpnode,6,[{file,"src/couch_btree.erl"},{line,535}]}, {couch_btree,modify_node,4,[{file,"src/couch_btree.erl
```

[error] 2022-05-24T15:19:54.944847Z nonode@nohost emulator -------- Error in process <0.12552.27> with exit value:

{invalid_compression,[{couch_compress,decompress,1,[{file,"src/couch_compress.erl"},{line,71}]},{couch_file,pread_term,2,[{file,"src/couch_file.erl"},{line,161}]},{couch_btree,get_node,2,[{file,"src/couch_btree.erl"},{line,435}]},{couch_btree,modify_node,4,[{file,"src/couch_btree.erl"},{line,388}]},{couch_btree,modify_kpnode,6,[{file,"src/couch_btree.erl"},{line,535}]},{couch_btree,modify_node,4,[{file,"src/couch_btree.erl"},{line,394}]},{couch_btree,modify_kpnode,6,[{file,"src/couch_btree.erl"},{line,535}]},{couch_btree,modify_node,4,[{file,"src/couch_btree.erl"},{line,394}]}]}

[error] 2022-05-24T15:19:54.945009Z nonode@nohost <0.12581.27> 34d336cfa8 rexi_server: from: nonode@nohost(<0.11656.27>) mfa: fabric_rpc:map_view/5 throw:{invalid_compression,[{couch_compress,decompress,1,[{file,"src/couch_compress.erl"},{line,71}]},{couch_file,pread_term,2,[{file,"src/couch_file.erl"},{line,161}]},{couch_btree,get_node,2,[{file,"src/couch_btree.erl"},{line,435}]},{couch_btree,modify_node,4,[{file,"src/couch_btree.erl"},{line,388}]},{couch_btree,modify_kpnode,6,[{file,"src/couch_btree.erl"},{line,535}]},{couch_btree,modify_node,4,[{file,"src/couch_btree.erl"},{line,394}]},{couch_btree,modify_kpnode,6,[{file,"src/couch_btree.erl"},{line,535}]},{couch_btree,modify_node,4,[{file,"src/couch_btree.erl"},{line,394}]}]} [{couch_mrview_util,get_view_index_state,5,[{file,"src/couch_mrview_util.erl"},{line,138}]},{couch_mrview_util,get_view,4,[{file,"src/couch_mrview_util.erl"},{line,82}]},{couch_mrview,query_view,6,[{file,"src/couch_mrview.erl"},{line,262}]},{rexi_server,init_p,3,[{file,"src/rexi_server.erl"},{line,138}]}]

[error] 2022-05-24T15:19:54.945332Z nonode@nohost <0.11656.27> 34d336cfa8 req_err(3722911048) invalid_compression : [{couch_compress,decompress,1,[{file,"src/couch_compress.erl"},{line,71}]},

Contributor guide

Open the contributing guide

Research direction

Start by reading src/couch_compress.erl at line 71 and the couch_file/pread_term path in src/couch_file.erl, then trace the couch_btree and couch_mrview calls shown in the error. Investigate the reported sensor_event database and CouchDB 3.1.1 Docker setup. Done means the invalid_compression cause is reproduced and a confirmed fix or recovery path is established.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, erlang, linux
Domain
databases
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.