requests_toolbelt.MultipartEncoder fails to encode S3File objects due to missing `len` property
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1k
- Forks
- 305
- Avg merge
- 22h 37m
- Merged PRs (30d)
- 4
Description
Hello,
I'm unable to use a S3File object together with requests_toolbelt.MultipartEncoder (https://toolbelt.readthedocs.io/en/latest/user.html). The MultipartEncoder expects the File object to contain len property to calculate its size and fails with the following:
File "/home/krzysztof/venvs/py311/lib/python3.11/site-packages/requests_toolbelt/multipart/encoder.py", line 125, in __init__
self._prepare_parts()
File "/home/krzysztof/venvs/py311/lib/python3.11/site-packages/requests_toolbelt/multipart/encoder.py", line 246, in _prepare_parts
self.parts = [Part.from_field(f, enc) for f in self._iter_fields()]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/krzysztof/venvs/py311/lib/python3.11/site-packages/requests_toolbelt/multipart/encoder.py", line 246, in <listcomp>
self.parts = [Part.from_field(f, enc) for f in self._iter_fields()]
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/krzysztof/venvs/py311/lib/python3.11/site-packages/requests_toolbelt/multipart/encoder.py", line 495, in from_field
return cls(headers, body)
^^^^^^^^^^^^^^^^^^
File "/home/krzysztof/venvs/py311/lib/python3.11/site-packages/requests_toolbelt/multipart/encoder.py", line 488, in __init__
self.len = len(self.headers) + total_len(self.body)
^^^^^^^^^^^^^^^^^^^^
File "/home/krzysztof/venvs/py311/lib/python3.11/site-packages/requests_toolbelt/multipart/encoder.py", line 432, in total_len
if hasattr(o, 'len'):
^^^^^^^^^^^^^^^^^
File "/home/krzysztof/venvs/py311/lib/python3.11/site-packages/requests_toolbelt/multipart/encoder.py", line 573, in len
return total_len(self.fd) - self.fd.tell()
~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~
TypeError: unsupported operand type(s) for -: 'NoneType' and 'int'
It works with a custom adapter like this:
class S3FileWithLenAdapter:
def __init__(self, s3_file):
self._s3_file = s3_file
@property
def len(self):
return self._s3_file.size
def __getattr__(self, name):
return getattr(self._s3_file, name)
Could you consider adding len property to S3File objects?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the S3File implementation and reviewing its existing size handling, then compare it with requests_toolbelt/multipart/encoder.py, especially total_len and the S3File-related len access shown in the traceback. Done means MultipartEncoder can consume an S3File without the reported TypeError; add or update focused tests if the repository provides them.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100