GoogleCloudPlatform / GoogleCloudPlatform/gsutil

AWS Fargate Task - Caught exception reading instance data

Open
#1,098 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
918
Forks
335
PR merge metrics
No merged PRs in 30d

Description

I am trying to run the following command in ECS Fargate Task
`./gsutil/gsutil -m rsync -r gs://$googleBucketPath s3://$awsBucketPath`

But It is throwing thet following error

> Caught exception reading instance data\nTraceback (most recent call last):\n File "/usr/lib/python3.6/urllib/request.py", line 1325, in do_open\n encode_chunked=req.has_header(\'Transfer-encoding\'))\n File "/usr/lib/python3.6/http/client.py", line 1264, in request\n self._send_request(method, url, body, headers, encode_chunked)\n File "/usr/lib/python3.6/http/client.py", line 1310, in _send_request\n self.endheaders(body, encode_chunked=encode_chunked)\n File "/usr/lib/python3.6/http/client.py", line 1259, in endheaders\n self._send_output(message_body, encode_chunked=encode_chunked)\n File "/usr/lib/python3.6/http/client.py", line 1038, in _send_output\n self.send(msg)\n File "/usr/lib/python3.6/http/client.py", line 976, in send\n self.connect()\n File "/usr/lib/python3.6/http/client.py", line 948, in connect\n (self.host,self.port), self.timeout, self.source_address)\n File "/usr/lib/python3.6/socket.py", line 724, in create_connection\n raise err\n File "/usr/lib/python3.6/socket.py", line 713, in create_connection\n sock.connect(sa)\nOSError: [Errno 22] Invalid argument\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/utils.py", line 220, in retry_url\n r = opener.open(req, timeout=timeout)\n File "/usr/lib/python3.6/urllib/request.py", line 526, in open\n response = self._open(req, data)\n File "/usr/lib/python3.6/urllib/request.py", line 544, in _open\n \'_open\', req)\n File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain\n result = func(*args)\n File "/usr/lib/python3.6/urllib/request.py", line 1353, in http_open\n return self.do_open(http.client.HTTPConnection, req)\n File "/usr/lib/python3.6/urllib/request.py", line 1327, in do_open\n raise URLError(err)\nurllib.error.URLError: \nERROR 0904 00:49:59.064282 utils.py] Unable to read instance data, giving up\nTraceback (most recent call last):\n File "/google-cloud-sdk/platform/gsutil/gsutil", line 21, in \n gsutil.RunMain()\n File "/google-cloud-sdk/platform/gsutil/gsutil.py", line 122, in RunMain\n sys.exit(gslib.__main__.main())\n File "/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 438, in main\n user_project=user_project)\n File "/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 767, in _RunNamedCommandAndHandleExceptions\n _HandleUnknownFailure(e)\n File "/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 633, in _RunNamedCommandAndHandleExceptions\n user_project=user_project)\n File "/google-cloud-sdk/platform/gsutil/gslib/command_runner.py", line 411, in RunNamedCommand\n return_code = command_inst.RunCommand()\n File "/google-cloud-sdk/platform/gsutil/gslib/commands/rsync.py", line 1626, in RunCommand\n dst_url = self._InsistContainer(self.args[1], True)\n File "/google-cloud-sdk/platform/gsutil/gslib/commands/rsync.py", line 1603, in _InsistContainer\n logger=self.logger)\n File "/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 1459, in ExpandUrlToSingleBlr\n for obj_or_prefix in list_iterator:\n File "/google-cloud-sdk/platform/gsutil/gslib/boto_translation.py", line 429, in ListObjects\n headers=headers)\n File "/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/storage_uri.py", line 166, in list_bucket\n bucket = self.get_bucket(headers=headers)\n File "/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/storage_uri.py", line 181, in get_bucket\n conn = self.connect()\n File "/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/storage_uri.py", line 117, in connect\n **connection_args)\n File "/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/s3/connection.py", line 205, in __init__\n validate_certs=validate_certs, profile_name=profile_name)\n File "/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/connection.py", line 573, in __init__\n host, config, self.provider, self._required_auth_capability())\n File "/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/auth.py", line 1032, in get_auth_handler\n \'Check your credentials\' % (len(names), str(names)))\nboto.exception.NoAuthHandlerFound: No handler was ready to authenticate. 1 handlers were checked. [\'S3HmacAuthV4Handler\'] Check your credentials\n'
> ERROR - 2020-09-04 00:49:59,176 - BaseDataIngestion.py - Exception: b'ERROR 0904 00:49:59.062589 utils.py] Caught exception reading instance data\nTraceback (most recent call last):\n File "/usr/lib/python3.6/urllib/request.py", line 1325, in do_open\n encode_chunked=req.has_header(\'Transfer-encoding\'))\n File "/usr/lib/python3.6/http/client.py", line 1264, in request\n self._send_request(method, url, body, headers, encode_chunked)\n File "/usr/lib/python3.6/http/client.py", line 1310, in _send_request\n self.endheaders(body, encode_chunked=encode_chunked)\n File "/usr/lib/python3.6/http/client.py", line 1259, in endheaders\n self._send_output(message_body, encode_chunked=encode_chunked)\n File "/usr/lib/python3.6/http/client.py", line 1038, in _send_output\n self.send(msg)\n File "/usr/lib/python3.6/http/client.py", line 976, in send\n self.connect()\n File "/usr/lib/python3.6/http/client.py", line 948, in connect\n (self.host,self.port), self.timeout, self.source_address)\n File "/usr/lib/python3.6/socket.py", line 724, in create_connection\n raise err\n File "/usr/lib/python3.6/socket.py", line 713, in create_connection\n sock.connect(sa)\nOSError: [Errno 22] Invalid argument\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/utils.py", line 220, in retry_url\n r = opener.open(req, timeout=timeout)\n File "/usr/lib/python3.6/urllib/request.py", line 526, in open\n response = self._open(req, data)\n File "/usr/lib/python3.6/urllib/request.py", line 544, in _open\n \'_open\', req)\n File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain\n result = func(*args)\n File "/usr/lib/python3.6/urllib/request.py", line 1353, in http_open\n return self.do_open(http.client.HTTPConnection, req)\n File "/usr/lib/python3.6/urllib/request.py", line 1327, in do_open\n raise URLError(err)\nurllib.error.URLError: \nERROR 0904 00:49:59.064282 utils.py] Unable to read instance data, giving up\nTraceback (most recent call last):\n File "/google-cloud-sdk/platform/gsutil/gsutil", line 21, in \n gsutil.RunMain()\n File "/google-cloud-sdk/platform/gsutil/gsutil.py", line 122, in RunMain\n sys.exit(gslib.__main__.main())\n File "/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 438, in main\n user_project=user_project)\n File "/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 767, in _RunNamedCommandAndHandleExceptions\n _HandleUnknownFailure(e)\n File "/google-cloud-sdk/platform/gsutil/gslib/__main__.py", line 633, in _RunNamedCommandAndHandleExceptions\n user_project=user_project)\n File "/google-cloud-sdk/platform/gsutil/gslib/command_runner.py", line 411, in RunNamedCommand\n return_code = command_inst.RunCommand()\n File "/google-cloud-sdk/platform/gsutil/gslib/commands/rsync.py", line 1626, in RunCommand\n dst_url = self._InsistContainer(self.args[1], True)\n File "/google-cloud-sdk/platform/gsutil/gslib/commands/rsync.py", line 1603, in _InsistContainer\n logger=self.logger)\n File "/google-cloud-sdk/platform/gsutil/gslib/utils/copy_helper.py", line 1459, in ExpandUrlToSingleBlr\n for obj_or_prefix in list_iterator:\n File "/google-cloud-sdk/platform/gsutil/gslib/boto_translation.py", line 429, in ListObjects\n headers=headers)\n File "/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/storage_uri.py", line 166, in list_bucket\n bucket = self.get_bucket(headers=headers)\n File "/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/storage_uri.py", line 181, in get_bucket\n conn = self.connect()\n File "/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/storage_uri.py", line 117, in connect\n **connection_args)\n File "/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/s3/connection.py", line 205, in __init__\n validate_certs=validate_certs, profile_name=profile_name)\n File "/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/connection.py", line 573, in __init__\n host, config, self.provider, self._required_auth_capability())\n File "/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/auth.py", line 1032, in get_auth_handler\n \'Check your credentials\' % (len(names), str(names)))\nboto.exception.NoAuthHandlerFound: No handler was ready to authenticate. 1 handlers were checked. [\'S3HmacAuthV4Handler\'] Check your credentials\n'

But when I run the same dockerized image in AWS EC2 Instance, it runs well.
I am new to this.
Kindly help me out :-)

Contributor guide

Open the contributing guide

Research direction

Reproduce the command in the AWS ECS Fargate environment and compare it with the working EC2 run. Start with the traceback locations in gslib/vendored/boto/boto/utils.py, gslib/vendored/boto/boto/auth.py, and gslib/commands/rsync.py; done means identifying the Fargate-specific failure and verifying a resolution or clearly documented workaround.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, docker, python
Domain
cli, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.