lbryio / lbryio/lbry-sdk

test_es_sync_utility: TypeError: argument should be bytes, buffer or ASCII string, not 'NoneType'

Open
#3,674 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area: tests
Dominant language
Python
Stars
7.2k
Forks
493
PR merge metrics
No merged PRs in 30d

Description

https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064

Transaction 'b01ca9a00c494dc971d0c0a1af3d32db242fd9c09a11b1a903b274234e7b9901' lookup is failing, and we get back None instead of raw TX bytes.

```
2022-09-22 12:49:48,178 - lbry.wallet.network - DEBUG - send blockchain.transaction.get_batch('b01ca9a00c494dc971d0c0a1af3d32db242fd9c09a11b1a903b274234e7b9901',) to localhost:50002 (30 timeout)
[6446](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6447)
2022-09-22 12:49:48,331 - lbry.wallet.network - DEBUG - response blockchain.transaction.get_batch('b01ca9a00c494dc971d0c0a1af3d32db242fd9c09a11b1a903b274234e7b9901',) from localhost:50002 (30 timeout) -> {'b01ca9a00c494dc971d0c0a1af3d32db242fd9c09a11b1a903b274234e7b9901': [None, {'block_height': -1}]}
[6447](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6448)
2022-09-22 12:49:48,333 - asyncio - ERROR - Task exception was never retrieved
[6448](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6449)
future: exception=TypeError("argument should be bytes, buffer or ASCII string, not 'NoneType'") created at /home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/tasks.py:16>
[6449](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6450)
source_traceback: Object created at (most recent call last):
[6450](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6451)
File "/home/runner/work/lbry-sdk/lbry-sdk/.tox/blockchain/bin/coverage", line 8, in
[6451](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6452)
sys.exit(main())
[6452](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6453)
File "/home/runner/work/lbry-sdk/lbry-sdk/.tox/blockchain/lib/python3.7/site-packages/coverage/cmdline.py", line 943, in main
[6453](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6454)
status = CoverageScript().command_line(argv)
[6454](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6455)
File "/home/runner/work/lbry-sdk/lbry-sdk/.tox/blockchain/lib/python3.7/site-packages/coverage/cmdline.py", line 659, in command_line
[6455](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6456)
return self.do_run(options, args)
[6456](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6457)
File "/home/runner/work/lbry-sdk/lbry-sdk/.tox/blockchain/lib/python3.7/site-packages/coverage/cmdline.py", line 830, in do_run
[6457](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6458)
runner.run()
[6458](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6459)
File "/home/runner/work/lbry-sdk/lbry-sdk/.tox/blockchain/lib/python3.7/site-packages/coverage/execfile.py", line 199, in run
[6459](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6460)
exec(code, main_mod.__dict__)
[6460](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6461)
File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/unittest/__main__.py", line 18, in
[6461](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6462)
main(module=None)
[6462](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6463)
File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/unittest/main.py", line 101, in __init__
[6463](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6464)
self.runTests()
[6464](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6465)
File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/unittest/main.py", line 271, in runTests
[6465](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6466)
self.result = testRunner.run(self.test)
[6466](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6467)
File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/unittest/runner.py", line 176, in run
[6467](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6468)
test(result)
[6468](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6469)
File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/unittest/suite.py", line 84, in __call__
[6469](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6470)
return self.run(*args, **kwds)
[6470](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6471)
File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/unittest/suite.py", line 122, in run
[6471](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6472)
test(result)
[6472](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6473)
File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/unittest/suite.py", line 84, in __call__
[6473](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6474)
return self.run(*args, **kwds)
[6474](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6475)
File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/unittest/suite.py", line 122, in run
[6475](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6476)
test(result)
[6476](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6477)
File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/unittest/suite.py", line 84, in __call__
[6477](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6478)
return self.run(*args, **kwds)
[6478](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6479)
File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/unittest/suite.py", line 122, in run
[6479](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6480)
test(result)
[6480](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6481)
File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/unittest/case.py", line 676, in __call__
[6481](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6482)
return self.run(*args, **kwds)
[6482](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6483)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/testcase.py", line 145, in run
[6483](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6484)
self.loop.run_until_complete(maybe_coroutine)
[6484](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6485)
File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py", line 574, in run_until_complete
[6485](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6486)
self.run_forever()
[6486](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6487)
File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
[6487](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6488)
self._run_once()
[6488](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6489)
File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py", line 1778, in _run_once
[6489](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6490)
handle._run()
[6490](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6491)
File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/events.py", line 88, in _run
[6491](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6492)
self._context.run(self._callback, *self._args)
[6492](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6493)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/rpc/session.py", line 433, in _handle_request
[6493](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6494)
result = await self.handle_request(request)
[6494](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6495)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/network.py", line 145, in handle_request
[6495](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6496)
controller.add(request.args)
[6496](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6497)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/stream.py", line 84, in add
[6497](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6498)
lambda subscription: None if skip else subscription._add(event)
[6498](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6499)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/stream.py", line 70, in _notify_and_ensure_future
[6499](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6500)
maybe_coroutine = notify(subscription)
[6500](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6501)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/stream.py", line 84, in
[6501](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6502)
lambda subscription: None if skip else subscription._add(event)
[6502](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6503)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/stream.py", line 32, in _add
[6503](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6504)
return self._on_data(data)
[6504](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6505)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/ledger.py", line 511, in process_status_update
[6505](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6506)
self._update_tasks.add(self.update_history(address, remote_status))
[6506](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6507)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/tasks.py", line 16, in add
[6507](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6508)
task = self._loop.create_task(coro)
[6508](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6509)
Traceback (most recent call last):
[6509](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6510)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/ledger.py", line 559, in update_history
[6510](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6511)
async for tx in self.request_synced_transactions(to_request, remote_history_txids, address):
[6511](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6512)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/ledger.py", line 667, in request_synced_transactions
[6512](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6513)
async for txs in self.request_transactions(((txid, height) for txid, height in to_request.values())):
[6513](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6514)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/ledger.py", line 660, in request_transactions
[6514](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6515)
txs = await self._single_batch(batch, remote_heights)
[6515](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6516)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/ledger.py", line 679, in _single_batch
[6516](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6517)
tx = Transaction(unhexlify(raw), height=remote_height)
[6517](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6518)
TypeError: argument should be bytes, buffer or ASCII string, not 'NoneType'
[6518](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:6519)```
```

This happens several times, then the test fails.

```
8274
======================================================================
[8275](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8276)
ERROR: test_es_sync_utility (integration.blockchain.test_wallet_server_sessions.TestESSync)
[8276](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8277)
----------------------------------------------------------------------
[8277](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8278)
Traceback (most recent call last):
[8278](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8279)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/testcase.py", line 145, in run
[8279](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8280)
self.loop.run_until_complete(maybe_coroutine)
[8280](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8281)
File "/opt/hostedtoolcache/Python/3.7.13/x64/lib/python3.7/asyncio/base_events.py", line 587, in run_until_complete
[8281](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8282)
return future.result()
[8282](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8283)
File "/home/runner/work/lbry-sdk/lbry-sdk/tests/integration/blockchain/test_wallet_server_sessions.py", line 144, in test_es_sync_utility
[8283](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8284)
self.assertEqual(11, len(await self.claim_search(order_by=['height'])))
[8284](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8285)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/testcase.py", line 708, in claim_search
[8285](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8286)
return (await self.out(self.daemon.jsonrpc_claim_search(**kwargs)))['items']
[8286](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8287)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/testcase.py", line 554, in out
[8287](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8288)
return json.loads(jsonrpc_dumps_pretty(await awaitable, ledger=self.ledger))['result']
[8288](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8289)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/extras/daemon/daemon.py", line 2644, in jsonrpc_claim_search
[8289](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8290)
txos, blocked, _, total = await self.ledger.claim_search(wallet.accounts, **kwargs)
[8290](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8291)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/ledger.py", line 909, in claim_search
[8291](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8292)
include_is_my_output=include_is_my_output
[8292](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8293)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/ledger.py", line 801, in _inflate_outputs
[8293](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8294)
async for tx in self.request_transactions(tuple(outputs.txs), cached=True):
[8294](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8295)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/ledger.py", line 660, in request_transactions
[8295](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8296)
txs = await self._single_batch(batch, remote_heights)
[8296](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8297)
File "/home/runner/work/lbry-sdk/lbry-sdk/lbry/wallet/ledger.py", line 679, in _single_batch
[8297](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8298)
tx = Transaction(unhexlify(raw), height=remote_height)
[8298](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8299)
TypeError: argument should be bytes, buffer or ASCII string, not 'NoneType'
[8299](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8300)

[8300](https://github.com/moodyjon/lbry-sdk/actions/runs/3105406501/jobs/5030990064#step:11:8301)
----------------------------------------------------------------------
```

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the test_es_sync_utility failure and trace the transaction lookup through lbry/wallet/ledger.py, especially update_history and request_synced_transactions. Read the logged blockchain.transaction.get_batch response and lbry/wallet/network.py handling to determine how a None transaction result reaches the sync utility. Done means the failing lookup no longer raises the reported TypeError and the test passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
blockchain
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.