algorand / algorand/go-algorand
arc28 and other log data in JSON format (may be) garbage
- 主要言語
- Go
- スター
- 1.4k
- フォーク
- 537
- 平均マージ
- 1日 6時間
- マージ済み PR(30日)
- 17
説明
### Subject of the issue
When using the `json` format on `/v2/blocks` it returns serialized `lg` data that is unusable / garbage in many cases.
In this case, the bad serialization is most easily identifiable by the visual presence of utf-8 replacement characters -> https://www.compart.com/en/unicode/U+FFFD
### Steps to reproduce
#### Method One
Look at `txn` index 16 (assuming start index of 0) which is a Tinyman swap:
https://mainnet-api.algonode.cloud/v2/blocks/41335116
Fields in `dt.ld.1` have garbage data along with `lg`, particularly logs like `output_asset_id` and `output_amount`
#### Method Two
Run this script to see a more thorough example of trying to access Tinyman log data, which is provided by `v2/blocks` in `json` format.
```python
from algosdk.v2client import algod
from base64 import b64decode
import pprint
algod_address = "https://mainnet-api.algonode.cloud"
algod_token = ""
algod_client = algod.AlgodClient(algod_token, algod_address)
def decode_logs(logs: "list") -> dict:
decoded_logs = dict()
for log in logs:
print("Decoding:", log)
if type(log) == str:
log = b64decode(log.encode())
if b"%i" in log:
i = log.index(b"%i")
s = log[0:i].decode()
value = int.from_bytes(log[i + 2 :], "big")
decoded_logs[s] = value
else:
raise NotImplementedError()
return decoded_logs
block = algod_client.block_info(block=41335116)
pp = pprint.PrettyPrinter(indent=2)
tinyman_tx = block.get('block').get('txns')[16]
log = tinyman_tx.get('dt').get('lg')
pp.pprint(log)
result = decode_logs(log)
pp.pprint(result)
```
### Thoughts
There are a few issues here:
1. JSON format is currently serializing data that may be unusable, wasting loads of bandwidth + cpu cycles
- Not a total waste in cases where the bytestream can be serialized
2. Switching to `msgpack` is replete with all sorts of quirks across various languages, the biggest being the main Javascript library can't even decode it without issues
3. @joe-p has mentioned:
> We technically could "fix" the JSON response to use surrogate escaping like Python supports, but again not every client would support that and that could actually break code that is currently working.
Which really sounds like the ideal answer, and then using path-versioning like `/v2.1/blocks` to provide a new endpoint to maintain legacy endpoint functioning.
4. @pbennett notes:
> problem likely stems from the 'logs' being defined as string not []byte - so it should be base64 encoded by algod for eg and its not.
コントリビューションガイド
評価
この issue はまだ評価されていません。