Recover/resync after wallet database corruption
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 7.2k
- Forks
- 493
- PR merge metrics
- No merged PRs in 30d
Description
## The Issue
A user ran into a database issue on mobile..this was after everything seemed to be working seemingly fine with new blocks being added. We should recover better from this problem. In this case, the database file had to be deleted and resynced.
```2019-03-31 22:17:29,408 INFO torba.client.baseledger:133: lbc_mainnet: added BlockHeightEvent(height=543671, change=1) header blocks, final height 543671
2019-03-31 22:21:51,141 INFO torba.client.baseledger:133: lbc_mainnet: added BlockHeightEvent(height=543672, change=1) header blocks, final height 543672
2019-03-31 22:23:11,568 INFO torba.client.baseledger:133: lbc_mainnet: added BlockHeightEvent(height=543673, change=1) header blocks, final height 543673
2019-03-31 22:23:30,726 INFO torba.client.baseledger:133: lbc_mainnet: added BlockHeightEvent(height=543674, change=1) header blocks, final height 543674
2019-03-31 22:24:42,108 INFO torba.client.baseledger:133: lbc_mainnet: added BlockHeightEvent(height=543675, change=1) header blocks, final height 543675
2019-03-31 22:26:29,184 INFO torba.client.baseledger:133: lbc_mainnet: added BlockHeightEvent(height=543676, change=1) header blocks, final height 543676
2019-03-31 22:28:34,875 INFO torba.client.baseledger:133: lbc_mainnet: added BlockHeightEvent(height=543677, change=1) header blocks, final height 543677
2019-03-31 22:30:21,440 ERROR lbrynet.extras.daemon.Daemon:496: error handling api request
Traceback (most recent call last):
File "/data/user/0/io.lbry.browser/files/app/crystax_python/site-packages/lbrynet/extras/daemon/Daemon.py", line 490, in _process_rpc_call
result = await result
File "/data/user/0/io.lbry.browser/files/app/crystax_python/site-packages/lbrynet/extras/daemon/Daemon.py", line 921, in jsonrpc_account_balance
dewies = await account.get_balance(confirmations=confirmations)
File "/data/user/0/io.lbry.browser/files/app/crystax_python/site-packages/torba/client/basedatabase.py", line 481, in get_balance
balance = await self.select_txos('SUM(amount)', **constraints)
File "/data/user/0/io.lbry.browser/files/app/crystax_python/site-packages/torba/client/basedatabase.py", line 438, in select_txos
" JOIN tx USING (txid)".format(cols), **constraints
File "/data/user/0/io.lbry.browser/files/app/crystax_python/stdlib.zip/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/data/user/0/io.lbry.browser/files/app/crystax_python/site-packages/torba/client/basedatabase.py", line 48, in __fetchall
return conn.execute(*args, **kwargs).fetchall()
sqlite3.OperationalError: disk I/O error
2019-03-31 22:30:24,912 INFO lbrynet.extras.daemon.Daemon:727: Get version info: {"processor": "", "python_version": "3.7.1", "platform": "Android 6.0 (API 23)", "os_release": "3.18.19+", "os_system": "Linux", "lbrynet_version": "0.32.4", "lbryschema_version": "0.0.16", "build": "release", "distro": {"id": "", "version": "", "version_parts": {"major": "", "minor": "", "build_number": ""}, "like": "", "codename": ""}, "desktop": "Unknown"}
2019-03-31 22:30:26,200 WARNING lbrynet.extras.daemon.Daemon:482: Extraneous parameters for resolve command: uris
2019-03-31 22:30:39,702 WARNING lbrynet.extras.daemon.Daemon:482: Extraneous parameters for resolve command: uris
2019-03-31 22:30:39,796 ERROR lbrynet.extras.daemon.Daemon:496: error handling api request
Traceback (most recent call last):
File "/data/user/0/io.lbry.browser/files/app/crystax_python/site-packages/lbrynet/extras/daemon/Daemon.py", line 490, in _process_rpc_call
result = await result
File "/data/user/0/io.lbry.browser/files/app/crystax_python/site-packages/lbrynet/extras/daemon/Daemon.py", line 921, in jsonrpc_account_balance
dewies = await account.get_balance(confirmations=confirmations)
File "/data/user/0/io.lbry.browser/files/app/crystax_python/site-packages/torba/client/basedatabase.py", line 481, in get_balance
balance = await self.select_txos('SUM(amount)', **constraints)
File "/data/user/0/io.lbry.browser/files/app/crystax_python/site-packages/torba/client/basedatabase.py", line 438, in select_txos
" JOIN tx USING (txid)".format(cols), **constraints
File "/data/user/0/io.lbry.browser/files/app/crystax_python/stdlib.zip/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/data/user/0/io.lbry.browser/files/app/crystax_python/site-packages/torba/client/basedatabase.py", line 48, in __fetchall
return conn.execute(*args, **kwargs).fetchall()
sqlite3.OperationalError: disk I/O error
2019-03-31 22:30:53,036 ERROR lbrynet.extras.daemon.Daemon:496: error handling api request
Traceback (most recent call last):
File "/data/user/0/io.lbry.browser/files/app/crystax_python/site-packages/lbrynet/extras/daemon/Daemon.py", line 490, in _process_rpc_call
result = await result
File "/data/user/0/io.lbry.browser/files/app/crystax_python/site-packages/lbrynet/extras/daemon/Daemon.py", line 921, in jsonrpc_account_balance
dewies = await account.get_balance(confirmations=confirmations)
File "/data/user/0/io.lbry.browser/files/app/crystax_python/site-packages/torba/client/basedatabase.py", line 481, in get_balance
balance = await self.select_txos('SUM(amount)', **constraints)
File "/data/user/0/io.lbry.browser/files/app/crystax_python/site-packages/torba/client/basedatabase.py", line 438, in select_txos
" JOIN tx USING (txid)".format(cols), **constraints
File "/data/user/0/io.lbry.browser/files/app/crystax_python/stdlib.zip/concurrent/futures/thread.py", line 57, in run
result = self.fn(*self.args, **self.kwargs)
File "/data/user/0/io.lbry.browser/files/app/crystax_python/site-packages/torba/client/basedatabase.py", line 48, in __fetchall
return conn.execute(*args, **kwargs).fetchall()
sqlite3.OperationalError: disk I/O error```
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing jsonrpc_account_balance in Daemon.py through get_balance and select_txos in basedatabase.py, using the reported sqlite3.OperationalError: disk I/O error as the failure case. Determine how database corruption is detected and how recovery or resync should be initiated; done means the failure no longer requires manually deleting the database file and resyncing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- android, python, sqlite
- Domain
- databases, mobile
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100