adafruit / adafruit/Adafruit_CircuitPython_Requests
Deceiving exception handling messages when available memory is low (below ~41k)
- Dominant language
- Python
- Stars
- 49
- Forks
- 39
- PR merge metrics
- No merged PRs in 30d
Description
I have run into issues with using the Requests library on a Pico W while getting JSON responses from an https url. The exception messages will vary based on the amount of free memory. I have created a quick [sample ](https://github.com/jake1164/requests_low_memory_example) to test and show the behavior.
The sample just loads images into a list and prints out the JSON returned until the board runs low on memory.
Below ~41k the exceptions make it look like a certification error and has the message 'Sending request failed'
Below ~27k it just fails and gives no exception message at all.
The same result happens with `requests.get(url)` and `requests.get(url, stream=True)`.
Crudely tested on:
CircuitPython 8.2.0, 8.1.0, 8.0.5, 8.0.2
Requests 1.14.1, 1.13.4, 1.12.9
```
before 13488
appended images 44160
appended image 44000
Getting json
getting url: https://api.weather.gov/alerts/active?area={state}
free memory 44000
free memory after 6224
{'correlationId': '606fcf6', 'status': 400, 'instance': 'https://api.weather.gov/requests/606fcf6', 'detail': 'Bad Request', 'type': 'https://api.weather.gov/problems/BadRequest', 'title': 'Bad Request', 'parameterErrors': [{'parameter': 'query.area[0]', 'message': 'Does not have a value in the enumeration ["AM","AN","GM","LC","LE","LH","LM","LO","LS","PH","PK","PM","PS","PZ","SL"]'}, {'parameter': 'query.area[0]', 'message': 'Does not have a value in the enumeration ["AL","AK","AS","AR","AZ","CA","CO","CT","DE","DC","FL","GA","GU","HI","ID","IL","IN","IA","KS","KY","LA","ME","MD","MA","MI","MN","MS","MO","MT","NE","NV","NH","NJ","NM","NY","NC","ND","OH","OK","OR","PA","PR","RI","SC","SD","TN","TX","UT","VT","VI","VA","WA","WV","WI","WY","MP","PW","FM","MH"]'}, {'parameter': 'query.area[0]', 'message': 'Failed to match exactly one schema'}]}
before 11184
appended images 41856
appended image 41696
Getting json
getting url: https://api.weather.gov/alerts/active?area={state}
free memory 41696
x509-crt-bundle:PK verify failed with error FFFFBD70
x509-crt-bundle:Failed to verify certificate
requests.get(url): Sending request failed
{}
...
before 32640
appended images 30336
appended image 30176
Getting json
getting url: https://api.weather.gov/alerts/active?area={state}
free memory 30176
requests.get(url): Sending request failed
{}
before 30336
appended images 28032
appended image 27872
Getting json
getting url: https://api.weather.gov/alerts/active?area={state}
free memory 27872
requests.get(url):
{}
```
Any secrets to getting this to work on low memory situations (like a pico driving a Matrix LED)?
Can we get better exception messages to indicate what is happening?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the linked low-memory reproduction and compare requests.get(url) with and without stream=True on CircuitPython. Reproduce the differing behavior around 41k and 27k free memory, then trace the resulting exception messages. Done means the failure no longer misleadingly resembles certificate validation or silently omits its cause, as agreed by the project maintainers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100