scribe "error while processing txs: int too large to convert" at mainnet block 1176266
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15
- Forks
- 16
- PR merge metrics
- No merged PRs in 30d
Description
Using the latest published docker image for `lbry/hub:master` (`lbry/hub@sha256:7ad1a2f570f061c9a8e2db65a9fe18f978c12df913f66ff93b3a49d87e674f6e`) the mainnet sync stops with this backtrace:
```
scribe_1 | 2023-03-26 07:12:43,994 INFO hub.service.BlockchainProcessorService:81: software version: 1.0.0
scribe_1 | 2023-03-26 07:12:43,994 INFO hub.service.BlockchainProcessorService:83: reorg limit is 200 blocks
scribe_1 | 2023-03-26 07:12:45,936 INFO hub.db.db:1036: opened db for writing: lbry-rocksdb (/database/lbry-rocksdb)
scribe_1 | 2023-03-26 07:12:45,937 INFO hub.db.db:1046: DB version: 12
scribe_1 | 2023-03-26 07:12:45,937 INFO hub.db.db:1047: coin: LBRY
scribe_1 | 2023-03-26 07:12:45,937 INFO hub.db.db:1048: network: mainnet
scribe_1 | 2023-03-26 07:12:45,937 INFO hub.db.db:1049: height: 1,176,266
scribe_1 | 2023-03-26 07:12:45,937 INFO hub.db.db:1050: tip: e5dbd99e5b626eac6e1f64cfc98c3030a9634b1a08c53e250ca33599d57ca015
scribe_1 | 2023-03-26 07:12:45,937 INFO hub.db.db:1051: tx count: 80,612,560
scribe_1 | 2023-03-26 07:12:45,937 INFO hub.db.db:1053: last indexed address statuses at block 1176266
scribe_1 | 2023-03-26 07:12:45,937 INFO hub.db.db:1054: using address status index: True
scribe_1 | 2023-03-26 07:12:45,937 INFO hub.db.db:1064: flush count: 1,176,267
scribe_1 | 2023-03-26 07:12:45,937 INFO hub.service.BlockchainProcessorService:86: initializing caches
scribe_1 | 2023-03-26 07:12:48,397 INFO hub.db.db:1168: populating header merkle cache...
scribe_1 | 2023-03-26 07:12:50,401 INFO hub.db.db:1173: header merkle cache populated in 2.0s
scribe_1 | 2023-03-26 07:12:50,401 INFO hub.service.BlockchainProcessorService:89: opened db at block 1176266
scribe_1 | 2023-03-26 07:12:50,420 INFO hub.scribe.prefetcher:82: catching up to daemon height 1,331,765 (155,499 blocks behind)
scribe_1 | 2023-03-26 07:12:53,447 ERROR hub.service.BlockchainProcessorService:255: advance blocks failed
scribe_1 | Traceback (most recent call last):
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 242, in check_and_advance_blocks
scribe_1 | txi_count, txo_count, c_added, c_spent, s_added, s_spent, abandoned, abandoned_chan = await self.run_in_thread_with_lock(
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 157, in run_in_thread_with_lock
scribe_1 | return await asyncio.shield(run_in_thread_locked())
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 156, in run_in_thread_locked
scribe_1 | return await asyncio.get_event_loop().run_in_executor(self._executor, func, *args)
scribe_1 | File "/usr/lib/python3.9/concurrent/futures/thread.py", line 52, in run
scribe_1 | result = self.fn(*self.args, **self.kwargs)
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 1726, in advance_block
scribe_1 | self._get_cumulative_update_ops(height)
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 1588, in _get_cumulative_update_ops
scribe_1 | self.db.prefix_db.future_effective_amount.stash_multi_put([
scribe_1 | File "/home/lbry/hub/db/interface.py", line 128, in stash_multi_put
scribe_1 | self._op_stack.stash_ops([RevertablePut(self.pack_key(*k), self.pack_value(*v)) for k, v in items])
scribe_1 | File "/home/lbry/hub/db/interface.py", line 128, in
scribe_1 | self._op_stack.stash_ops([RevertablePut(self.pack_key(*k), self.pack_value(*v)) for k, v in items])
scribe_1 | File "/home/lbry/hub/db/prefixes.py", line 1844, in pack_value
scribe_1 | return super().pack_value(future_effective_amount)
scribe_1 | File "/home/lbry/hub/db/interface.py", line 160, in pack_value
scribe_1 | return cls.value_struct.pack(*args)
scribe_1 | struct.error: int too large to convert
scribe_1 | 2023-03-26 07:12:53,449 ERROR hub.service.BlockchainProcessorService:2106: error while processing txs: int too large to convert
scribe_1 | Traceback (most recent call last):
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 2102, in process_blocks_and_mempool_forever
scribe_1 | await self.check_and_advance_blocks(blocks)
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 242, in check_and_advance_blocks
scribe_1 | txi_count, txo_count, c_added, c_spent, s_added, s_spent, abandoned, abandoned_chan = await self.run_in_thread_with_lock(
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 157, in run_in_thread_with_lock
scribe_1 | return await asyncio.shield(run_in_thread_locked())
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 156, in run_in_thread_locked
scribe_1 | return await asyncio.get_event_loop().run_in_executor(self._executor, func, *args)
scribe_1 | File "/usr/lib/python3.9/concurrent/futures/thread.py", line 52, in run
scribe_1 | result = self.fn(*self.args, **self.kwargs)
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 1726, in advance_block
scribe_1 | self._get_cumulative_update_ops(height)
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 1588, in _get_cumulative_update_ops
scribe_1 | self.db.prefix_db.future_effective_amount.stash_multi_put([
scribe_1 | File "/home/lbry/hub/db/interface.py", line 128, in stash_multi_put
scribe_1 | self._op_stack.stash_ops([RevertablePut(self.pack_key(*k), self.pack_value(*v)) for k, v in items])
scribe_1 | File "/home/lbry/hub/db/interface.py", line 128, in
scribe_1 | self._op_stack.stash_ops([RevertablePut(self.pack_key(*k), self.pack_value(*v)) for k, v in items])
scribe_1 | File "/home/lbry/hub/db/prefixes.py", line 1844, in pack_value
scribe_1 | return super().pack_value(future_effective_amount)
scribe_1 | File "/home/lbry/hub/db/interface.py", line 160, in pack_value
scribe_1 | return cls.value_struct.pack(*args)
scribe_1 | struct.error: int too large to convert
scribe_1 | 2023-03-26 07:12:53,450 ERROR hub.service.BlockchainProcessorService:2114: fatal error in block processor loop: int too large to convert
scribe_1 | Traceback (most recent call last):
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 2107, in process_blocks_and_mempool_forever
scribe_1 | raise err
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 2102, in process_blocks_and_mempool_forever
scribe_1 | await self.check_and_advance_blocks(blocks)
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 242, in check_and_advance_blocks
scribe_1 | txi_count, txo_count, c_added, c_spent, s_added, s_spent, abandoned, abandoned_chan = await self.run_in_thread_with_lock(
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 157, in run_in_thread_with_lock
scribe_1 | return await asyncio.shield(run_in_thread_locked())
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 156, in run_in_thread_locked
scribe_1 | return await asyncio.get_event_loop().run_in_executor(self._executor, func, *args)
scribe_1 | File "/usr/lib/python3.9/concurrent/futures/thread.py", line 52, in run
scribe_1 | result = self.fn(*self.args, **self.kwargs)
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 1726, in advance_block
scribe_1 | self._get_cumulative_update_ops(height)
scribe_1 | File "/home/lbry/hub/scribe/service.py", line 1588, in _get_cumulative_update_ops
scribe_1 | self.db.prefix_db.future_effective_amount.stash_multi_put([
scribe_1 | File "/home/lbry/hub/db/interface.py", line 128, in stash_multi_put
scribe_1 | self._op_stack.stash_ops([RevertablePut(self.pack_key(*k), self.pack_value(*v)) for k, v in items])
scribe_1 | File "/home/lbry/hub/db/interface.py", line 128, in
scribe_1 | self._op_stack.stash_ops([RevertablePut(self.pack_key(*k), self.pack_value(*v)) for k, v in items])
scribe_1 | File "/home/lbry/hub/db/prefixes.py", line 1844, in pack_value
scribe_1 | return super().pack_value(future_effective_amount)
scribe_1 | File "/home/lbry/hub/db/interface.py", line 160, in pack_value
scribe_1 | return cls.value_struct.pack(*args)
scribe_1 | struct.error: int too large to convert
```
Please let me know what other info I can provide (I'm reasonably experienced with Python and docker, I could set up a local build to dig into it if needed).
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the mainnet sync failure using the published lbry/hub Docker image and the block height shown in the report. Start in scribe/service.py at _get_cumulative_update_ops, then follow packing through db/interface.py and db/prefixes.py; done means block processing advances without the int conversion error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python
- Domain
- blockchain
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100