oracle / oracle/oci-python-sdk
MissingHeaderBodySeparatorDefect when uploading items to OSS
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 474
- Forks
- 321
- Avg merge
- 23m
- Merged PRs (30d)
- 4
Description
When uploading files to OSS, I randomly (~1000 items in) experience this error after I run
with open(file_path, 'rb') as f:
obj_data = f.read()
try:
response = oci_client.put_object(
OSS_NAMESPACE,
OSS_BUCKET_NAME,
oss_object_name, # object name
obj_data, # object body (bytes)
)
if response.status != 200:
msg = f'WARNING: {oss_object_name} upload resulted in status {response.status}. {response.data}'
logging.log(logging.WARN, msg)
print(msg)
else:
msg = f'SUCCESS! {oss_object_name} uploaded'
logging.log(logging.INFO, msg)
print(msg)
except oci.exceptions.ServiceError as e:
# Handling OCI specific exceptions
msg = f'ERROR: {oss_object_name} upload failed with ServiceError. {e.status} {e.message}'
logging.error(msg)
print(msg)
except Exception as e:
msg = f'ERROR: {oss_object_name} upload failed. {e}'
logging.log(logging.ERROR, msg)
print(msg)
Here's the traceback:
2024-02-19 20:09:39,991:WARNING:Failed to parse headers (url=https://objectstorage.ap-osaka-1.oraclecloud.com:443/n/ax2tjlcefnht/b/laion2B-en-aesthetic/o/000000401/000000401.txt): [MissingHeaderBodySeparatorDefect()], unparsed data: 'HTTP/1.1 200 OK\r\netag: 07f8f92b-fd96-49ec-b9db-1b927e144116\r\nlast-modified: Mon, 19 Feb 2024 20:09:39 GMT\r\nopc-content-md5: tMGI50PK84P9//D2pQ8evQ==\r\nversion-id: 04303300-a3ec-4fdb-a2eb-58cd65effb06\r\nContent-Length: 0\r\ndate: Mon, 19 Feb 2024 20:09:39 GMT\r\nopc-request-id: kix-1:BaX2PWrWLWStJYPKMWmC_TRLmcJaINqwJQey95jbJrG80yzz4rVgEBGGUeet-gvk\r\nx-api-id: native\r\nx-content-type-options: nosniff\r\nstrict-transport-security: max-age=31536000; includeSubDomains\r\naccess-control-allow-origin: *\r\naccess-control-allow-methods: POST,PUT,GET,HEAD,DELETE,OPTIONS\r\naccess-control-allow-credentials: true\r\naccess-control-expose-headers: access-control-allow-credentials,access-control-allow-methods,access-control-allow-origin,content-length,date,etag,last-modified,opc-client-info,opc-content-md5,opc-request-id,strict-transport-security,version-id,x-api-id,x-content-type-options\r\n\r\n'
Traceback (most recent call last):
File "/home/Mindy/.cache/pypoetry/virtualenvs/t2i-adapter-3QVI5i9t-py3.10/lib/python3.10/site-packages/oci/_vendor/urllib3/connectionpool.py", line 474, in _make_request
assert_header_parsing(httplib_response.msg)
File "/home/Mindy/.cache/pypoetry/virtualenvs/t2i-adapter-3QVI5i9t-py3.10/lib/python3.10/site-packages/oci/_vendor/urllib3/util/response.py", line 96, in assert_header_parsing
raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)
oci._vendor.urllib3.exceptions.HeaderParsingError: [MissingHeaderBodySeparatorDefect()], unparsed data: 'HTTP/1.1 200 OK\r\netag: 07f8f92b-fd96-49ec-b9db-1b927e144116\r\nlast-modified: Mon, 19 Feb 2024 20:09:39 GMT\r\nopc-content-md5: tMGI50PK84P9//D2pQ8evQ==\r\nversion-id: 04303300-a3ec-4fdb-a2eb-58cd65effb06\r\nContent-Length: 0\r\ndate: Mon, 19 Feb 2024 20:09:39 GMT\r\nopc-request-id: kix-1:BaX2PWrWLWStJYPKMWmC_TRLmcJaINqwJQey95jbJrG80yzz4rVgEBGGUeet-gvk\r\nx-api-id: native\r\nx-content-type-options: nosniff\r\nstrict-transport-security: max-age=31536000; includeSubDomains\r\naccess-control-allow-origin: *\r\naccess-control-allow-methods: POST,PUT,GET,HEAD,DELETE,OPTIONS\r\naccess-control-allow-credentials: true\r\naccess-control-expose-headers: access-control-allow-credentials,access-control-allow-methods,access-control-allow-origin,content-length,date,etag,last-modified,opc-client-info,opc-content-md5,opc-request-id,strict-transport-security,version-id,x-api-id,x-content-type-options\r\n\r\n'
Even after adding try...except, I can't catch the exception and my code hangs. put_object worked fine for me two weeks ago when I uploaded hundreds of thousands of items.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the reported put_object call and the traceback entry points in oci/_vendor/urllib3/connectionpool.py and oci/_vendor/urllib3/util/response.py. Reproduce the failure during repeated OSS uploads, determine why the response triggers MissingHeaderBodySeparatorDefect and hangs outside the shown try/except, then verify that the failure is surfaced reliably without disrupting successful uploads.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100