boxmein / boxmein/lunchbot

UTF8 characters at 8192 byte boundary will cause errors

Open
#1 0 comments 0 reactions 1 assignee Claimed by @boxmein View on GitHub
Dominant language
Python
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

```
'utf-8' codec can't decode byte 0xc3 in position 8191: unexpected end of data: UnicodeDecodeError
Traceback (most recent call last):
File "/var/task/serverless_sdk/__init__.py", line 54, in wrapped_handler
return user_handler(event, context)
File "/var/task/handler.py", line 162, in slack
offers = get_lunch_offers(city)
File "/var/task/handler.py", line 100, in get_lunch_offers
offers = streaming_download_and_parse_offers(url)
File "/var/task/handler.py", line 89, in streaming_download_and_parse_offers
res_data = response.read(8192).decode('utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xc3 in position 8191: unexpected end of data
```

My assumption is that there's a UTF8 character consisting of multiple bytes at the boundary of the byte array. The fix would be to read 8192 bytes, try decode, if fails, read 8192 mode and retry.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.