oracle / oracle/oci-python-sdk

MissingHeaderBodySeparatorDefect when uploading items to OSS

Open
#622 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

SDK
Dominant language
Python
Stars
474
Forks
321
Avg merge
23m
Merged PRs (30d)
4

Description

When uploading files to OSS, I randomly (~1000 items in) experience this error after I run

with open(file_path, 'rb') as f:
                obj_data = f.read()
                try:
                      response = oci_client.put_object(
                                    OSS_NAMESPACE, 
                                    OSS_BUCKET_NAME,
                                    oss_object_name,  # object name 
                                    obj_data, # object body (bytes)
                                )
                       if response.status != 200:
                              msg = f'WARNING: {oss_object_name} upload resulted in status {response.status}. {response.data}'
                              logging.log(logging.WARN, msg)
                              print(msg)
                        else:
                              msg = f'SUCCESS! {oss_object_name} uploaded'
                              logging.log(logging.INFO, msg)
                              print(msg)
                except oci.exceptions.ServiceError as e:
                    # Handling OCI specific exceptions
                    msg = f'ERROR: {oss_object_name} upload failed with ServiceError. {e.status} {e.message}'
                    logging.error(msg)
                    print(msg)
                except Exception as e:
                    msg = f'ERROR: {oss_object_name} upload failed. {e}'
                    logging.log(logging.ERROR, msg)
                    print(msg)

Here's the traceback:

2024-02-19 20:09:39,991:WARNING:Failed to parse headers (url=https://objectstorage.ap-osaka-1.oraclecloud.com:443/n/ax2tjlcefnht/b/laion2B-en-aesthetic/o/000000401/000000401.txt): [MissingHeaderBodySeparatorDefect()], unparsed data: 'HTTP/1.1 200 OK\r\netag: 07f8f92b-fd96-49ec-b9db-1b927e144116\r\nlast-modified: Mon, 19 Feb 2024 20:09:39 GMT\r\nopc-content-md5: tMGI50PK84P9//D2pQ8evQ==\r\nversion-id: 04303300-a3ec-4fdb-a2eb-58cd65effb06\r\nContent-Length: 0\r\ndate: Mon, 19 Feb 2024 20:09:39 GMT\r\nopc-request-id: kix-1:BaX2PWrWLWStJYPKMWmC_TRLmcJaINqwJQey95jbJrG80yzz4rVgEBGGUeet-gvk\r\nx-api-id: native\r\nx-content-type-options: nosniff\r\nstrict-transport-security: max-age=31536000; includeSubDomains\r\naccess-control-allow-origin: *\r\naccess-control-allow-methods: POST,PUT,GET,HEAD,DELETE,OPTIONS\r\naccess-control-allow-credentials: true\r\naccess-control-expose-headers: access-control-allow-credentials,access-control-allow-methods,access-control-allow-origin,content-length,date,etag,last-modified,opc-client-info,opc-content-md5,opc-request-id,strict-transport-security,version-id,x-api-id,x-content-type-options\r\n\r\n'
Traceback (most recent call last):
  File "/home/Mindy/.cache/pypoetry/virtualenvs/t2i-adapter-3QVI5i9t-py3.10/lib/python3.10/site-packages/oci/_vendor/urllib3/connectionpool.py", line 474, in _make_request
    assert_header_parsing(httplib_response.msg)
  File "/home/Mindy/.cache/pypoetry/virtualenvs/t2i-adapter-3QVI5i9t-py3.10/lib/python3.10/site-packages/oci/_vendor/urllib3/util/response.py", line 96, in assert_header_parsing
    raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)
oci._vendor.urllib3.exceptions.HeaderParsingError: [MissingHeaderBodySeparatorDefect()], unparsed data: 'HTTP/1.1 200 OK\r\netag: 07f8f92b-fd96-49ec-b9db-1b927e144116\r\nlast-modified: Mon, 19 Feb 2024 20:09:39 GMT\r\nopc-content-md5: tMGI50PK84P9//D2pQ8evQ==\r\nversion-id: 04303300-a3ec-4fdb-a2eb-58cd65effb06\r\nContent-Length: 0\r\ndate: Mon, 19 Feb 2024 20:09:39 GMT\r\nopc-request-id: kix-1:BaX2PWrWLWStJYPKMWmC_TRLmcJaINqwJQey95jbJrG80yzz4rVgEBGGUeet-gvk\r\nx-api-id: native\r\nx-content-type-options: nosniff\r\nstrict-transport-security: max-age=31536000; includeSubDomains\r\naccess-control-allow-origin: *\r\naccess-control-allow-methods: POST,PUT,GET,HEAD,DELETE,OPTIONS\r\naccess-control-allow-credentials: true\r\naccess-control-expose-headers: access-control-allow-credentials,access-control-allow-methods,access-control-allow-origin,content-length,date,etag,last-modified,opc-client-info,opc-content-md5,opc-request-id,strict-transport-security,version-id,x-api-id,x-content-type-options\r\n\r\n'

Even after adding try...except, I can't catch the exception and my code hangs. put_object worked fine for me two weeks ago when I uploaded hundreds of thousands of items.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the reported put_object call and the traceback entry points in oci/_vendor/urllib3/connectionpool.py and oci/_vendor/urllib3/util/response.py. Reproduce the failure during repeated OSS uploads, determine why the response triggers MissingHeaderBodySeparatorDefect and hangs outside the shown try/except, then verify that the failure is surfaced reliably without disrupting successful uploads.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.