swagger-api / swagger-api/swagger-codegen
[python] Binary string response encoding with Python 3
Nobody has claimed this yet.
- Dominant language
- Mustache
- Stars
- 17.8k
- Forks
- 6k
- PR merge metrics
- No merged PRs in 30d
Description
When using binary string responses under Python 3, the encoding defaults to UTF8 which is incompatible with binary data. This looks to stem from Python 3 having str default to UTF8 and a new type called bytes which represents binary data. This is a significant departure from functionality in Python 2 and you can read more about it.
This issue does not exist in Python 2.
Swagger Revision: 18262c1b6132a28438e2c3d9426458c413f011f8 (master as of right now)
Response schema:
schema:
type: string
format: binary
Example Error:
Traceback (most recent call last):
File "./dr.py", line 13, in <module>
"document_type": "pdf", # pdf or xls or xlsx
File "/usr/local/lib/python3.5/site-packages/docraptor/apis/doc_api.py", line 203, in create_doc
callback=params.get('callback'))
File "/usr/local/lib/python3.5/site-packages/docraptor/api_client.py", line 322, in call_api
response_type, auth_settings, callback)
File "/usr/local/lib/python3.5/site-packages/docraptor/api_client.py", line 149, in __call_api
post_params=post_params, body=body)
File "/usr/local/lib/python3.5/site-packages/docraptor/api_client.py", line 358, in request
body=body)
File "/usr/local/lib/python3.5/site-packages/docraptor/rest.py", line 208, in POST
body=body)
File "/usr/local/lib/python3.5/site-packages/docraptor/rest.py", line 171, in request
r.data = r.data.decode('utf8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe2 in position 10: invalid continuation byte
I'm not sure what a good general fix would look like but I have a work-around for our use by delaying byte decoding until right before JSON deserialization and short-circuiting str. You can see my fix directly on generated code.
Related Issues and PRs:
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in generated rest.py at the request path that assigns r.data = r.data.decode('utf8'), then inspect api_client.py where response types are handled. Compare the Python 3 binary schema case with JSON deserialization and use the linked generated-code workaround as context. Done means binary responses no longer undergo incompatible UTF-8 decoding while normal JSON responses still work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100