bitshares / bitshares/bitshares-core

Missing account history data in ElasticSearch if node is restarted while syncing

Open
#2,693 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
1.2k
Forks
660
Avg merge
8h 17m
Merged PRs (30d)
26

Description

**Bug Description**

The `elasticsearch` plugin does not send data in memory to ElasticSearch when the node is shutting down. If the node is still syncing (30+ seconds behind the current head block time), after it is restarted, it will resume from a potential much later block number.

Related:
* If the node is already in sync, the `elasticsearch` plugin sends data to ElasticSearch on every block, so no issue.
* The missing data can be fixed with a replay with an appropriate starting block number.
* This issue does not exist in the `es_objects` plugin (version `test-6.1.2`).

https://github.com/blocksights/blocksights-open-explorer/issues/7 might be caused by this issue.

This might be the answer to https://github.com/bitshares/bitshares-core/issues/2549#issuecomment-991921614
> ... it is strange that the error didn't pop up on the first sync, but popped up on replay.

https://github.com/bitshares/bitshares-core/issues/2493 is also somewhat related.

**Impacts**
Describe which portion(s) of BitShares Core may be impacted by this bug. Please tick at least one box.
- [ ] API (the application programming interface)
- [ ] Build (the build process or something prior to compiled code)
- [ ] CLI (the command line wallet)
- [ ] Deployment (the deployment process after building such as Docker, Travis, etc.)
- [ ] DEX (the Decentralized EXchange, market engine, etc.)
- [ ] P2P (the peer-to-peer network for transaction/block propagation)
- [ ] Performance (system or user efficiency, etc.)
- [ ] Protocol (the blockchain logic, consensus, validation, etc.)
- [ ] Security (the security of system or user data, etc.)
- [ ] UX (the User Experience)
- [x] Other (please add below)
- ElasticSearch History plugin

**Host Environment**
Please provide details about the host environment. Much of this information can be found running: `witness_node --version`.
- BitShares Version: `test-6.1.2`

## CORE TEAM TASK LIST
- [ ] Evaluate / Prioritize Bug Report
- [ ] Refine User Stories / Requirements
- [ ] Define Test Cases
- [ ] Design / Develop Solution
- [ ] Perform QA/Testing
- [ ] Update Documentation

Contributor guide

No contributing guide indexed for this repository

Research direction

Locate the elasticsearch plugin's block-sync and node-shutdown entry points; no file or test is named in the report, so begin by tracing where in-memory history data is handled during shutdown. Reproduce a restart while syncing more than 30 seconds behind, then verify that pending data is sent and no block range is skipped after restart.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, elasticsearch
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.