linear_learner_multi_model_endpoint_inf_pipeline.ipynb ReadTimeoutError

https://github.com/awslabs/amazon-sagemaker-examples/blob/master/advanced_functionality/multi_model_linear_learner_home_value/linear_learner_multi_model_endpoint_inf_pipeline.ipynb

When running this:
```
##Starting with no models loaded into the container
##Invoke the first 100 models
invoke_multiple_models_mme(0, 100)
```
It errors out eventually:
```
Using model Houston_TX/0_Houston_TX.tar.gz to predict price of this house: [1995, 3230.9862244886135, 6, 1.5, 1.05, 2, 'y', 'n']
$514,362.44, took 1,310 ms

Using model Houston_TX/1_Houston_TX.tar.gz to predict price of this house: [1982, 2788.5213656009223, 2, 2.5, 0.91, 2, 'n', 'y']
$337,812.78, took 1,253 ms

Using model Houston_TX/2_Houston_TX.tar.gz to predict price of this house: [2003, 1395.6583882967261, 3, 1.0, 0.94, 2, 'n', 'y']
$221,539.23, took 1,269 ms

Using model Houston_TX/3_Houston_TX.tar.gz to predict price of this house: [1987, 4246.743774104067, 2, 3.0, 0.66, 3, 'y', 'n']
$620,207.75, took 1,249 ms

Using model Houston_TX/4_Houston_TX.tar.gz to predict price of this house: [1993, 3298.9823098047173, 3, 1.5, 1.24, 1, 'y', 'y']
$476,088.16, took 1,281 ms

Using model Houston_TX/5_Houston_TX.tar.gz to predict price of this house: [1991, 3243.140357145821, 5, 1.5, 0.49, 0, 'n', 'n']
$422,990.88, took 1,274 ms

Using model Houston_TX/6_Houston_TX.tar.gz to predict price of this house: [1980, 3052.5670370012817, 3, 1.0, 1.28, 0, 'n', 'n']
$333,735.97, took 1,247 ms

Using model Houston_TX/7_Houston_TX.tar.gz to predict price of this house: [1993, 3201.823240499402, 3, 1.0, 0.85, 2, 'y', 'n']
$463,544.09, took 1,247 ms

Using model Houston_TX/8_Houston_TX.tar.gz to predict price of this house: [2007, 3305.37730193126, 6, 3.0, 1.52, 0, 'n', 'y']
$560,165.44, took 1,260 ms

Using model Houston_TX/9_Houston_TX.tar.gz to predict price of this house: [2019, 3397.9237509278264, 4, 1.5, 0.49, 0, 'y', 'n']
$595,918.12, took 1,267 ms

Using model Houston_TX/10_Houston_TX.tar.gz to predict price of this house: [1993, 1805.165307601194, 3, 1.5, 1.16, 0, 'n', 'n']
$214,172.16, took 1,289 ms

Using model Houston_TX/11_Houston_TX.tar.gz to predict price of this house: [1988, 4148.094755853439, 3, 2.5, 1.27, 1, 'y', 'n']
$593,605.50, took 1,269 ms

Using model Houston_TX/12_Houston_TX.tar.gz to predict price of this house: [2003, 3946.525774785787, 5, 3.0, 0.91, 2, 'y', 'y']
$667,916.75, took 1,289 ms

Using model Houston_TX/13_Houston_TX.tar.gz to predict price of this house: [1986, 2961.395337564516, 6, 3.0, 1.23, 3, 'n', 'n']
$448,151.53, took 1,250 ms

Using model Houston_TX/14_Houston_TX.tar.gz to predict price of this house: [1995, 3686.8199059718463, 6, 3.0, 1.23, 2, 'y', 'n']
$606,144.12, took 1,279 ms

Using model Houston_TX/15_Houston_TX.tar.gz to predict price of this house: [1986, 2922.8723344806026, 3, 3.0, 0.36, 0, 'n', 'n']
$349,838.97, took 1,258 ms

Using model Houston_TX/16_Houston_TX.tar.gz to predict price of this house: [1982, 1272.2873756178415, 3, 1.0, 0.78, 0, 'y', 'n']
$86,451.22, took 1,251 ms

Using model Houston_TX/17_Houston_TX.tar.gz to predict price of this house: [1996, 2251.0885612583725, 5, 2.5, 0.75, 3, 'n', 'n']
$365,141.25, took 1,250 ms

Using model Houston_TX/18_Houston_TX.tar.gz to predict price of this house: [1992, 2139.837251830015, 5, 2.5, 0.92, 2, 'y', 'y']
$336,464.44, took 1,262 ms

Using model Houston_TX/19_Houston_TX.tar.gz to predict price of this house: [1984, 2908.203980468678, 5, 2.5, 0.65, 0, 'n', 'y']
$353,844.56, took 1,248 ms

Using model Houston_TX/20_Houston_TX.tar.gz to predict price of this house: [1998, 2085.314708177354, 6, 1.5, 1.04, 1, 'y', 'n']
---------------------------------------------------------------------------
timeout Traceback (most recent call last)
~/anaconda3/envs/python3/lib/python3.6/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
420 # Otherwise it looks like a bug in the code.
--> 421 six.raise_from(e, None)
422 except (SocketTimeout, BaseSSLError, SocketError) as e:

~/anaconda3/envs/python3/lib/python3.6/site-packages/urllib3/packages/six.py in raise_from(value, from_value)

~/anaconda3/envs/python3/lib/python3.6/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
415 try:
--> 416 httplib_response = conn.getresponse()
417 except BaseException as e:

~/anaconda3/envs/python3/lib/python3.6/http/client.py in getresponse(self)
1353 try:
-> 1354 response.begin()
1355 except ConnectionError:

~/anaconda3/envs/python3/lib/python3.6/http/client.py in begin(self)
306 while True:
--> 307 version, status, reason = self._read_status()
308 if status != CONTINUE:

~/anaconda3/envs/python3/lib/python3.6/http/client.py in _read_status(self)
267 def _read_status(self):
--> 268 line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
269 if len(line) > _MAXLINE:

~/anaconda3/envs/python3/lib/python3.6/socket.py in readinto(self, b)
585 try:
--> 586 return self._sock.recv_into(b)
587 except timeout:

~/anaconda3/envs/python3/lib/python3.6/ssl.py in recv_into(self, buffer, nbytes, flags)
1011 self.__class__)
-> 1012 return self.read(nbytes, buffer)
1013 else:

~/anaconda3/envs/python3/lib/python3.6/ssl.py in read(self, len, buffer)
873 try:
--> 874 return self._sslobj.read(len, buffer)
875 except SSLError as x:

~/anaconda3/envs/python3/lib/python3.6/ssl.py in read(self, len, buffer)
630 if buffer is not None:
--> 631 v = self._sslobj.read(len, buffer)
632 else:

timeout: The read operation timed out

During handling of the above exception, another exception occurred:

ReadTimeoutError Traceback (most recent call last)
~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/httpsession.py in send(self, request)
262 decode_content=False,
--> 263 chunked=self._chunked(request.headers),
264 )

~/anaconda3/envs/python3/lib/python3.6/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
719 retries = retries.increment(
--> 720 method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
721 )

~/anaconda3/envs/python3/lib/python3.6/site-packages/urllib3/util/retry.py in increment(self, method, url, response, error, _pool, _stacktrace)
375 # Disabled, indicate to re-raise the error.
--> 376 raise six.reraise(type(error), error, _stacktrace)
377

~/anaconda3/envs/python3/lib/python3.6/site-packages/urllib3/packages/six.py in reraise(tp, value, tb)
734 raise value.with_traceback(tb)
--> 735 raise value
736 finally:

~/anaconda3/envs/python3/lib/python3.6/site-packages/urllib3/connectionpool.py in urlopen(self, method, url, body, headers, retries, redirect, assert_same_host, timeout, pool_timeout, release_conn, chunked, body_pos, **response_kw)
671 headers=headers,
--> 672 chunked=chunked,
673 )

~/anaconda3/envs/python3/lib/python3.6/site-packages/urllib3/connectionpool.py in _make_request(self, conn, method, url, timeout, chunked, **httplib_request_kw)
422 except (SocketTimeout, BaseSSLError, SocketError) as e:
--> 423 self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
424 raise

~/anaconda3/envs/python3/lib/python3.6/site-packages/urllib3/connectionpool.py in _raise_timeout(self, err, url, timeout_value)
330 raise ReadTimeoutError(
--> 331 self, url, "Read timed out. (read timeout=%s)" % timeout_value
332 )

ReadTimeoutError: AWSHTTPSConnectionPool(host='runtime.sagemaker.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=80)

During handling of the above exception, another exception occurred:

ReadTimeoutError Traceback (most recent call last)
in
1 ##Starting with no models loaded into the container
2 ##Invoke the first 100 models
----> 3 invoke_multiple_models_mme(0, 100)

in invoke_multiple_models_mme(model_range_low, model_range_high)
4 new_model_name="{}_{}".format(i, model_name)
5 full_model_name = '{}/{}.tar.gz'.format(model_name, new_model_name)
----> 6 predict_one_house_value(gen_random_house()[:-1], full_model_name, predictor)

in predict_one_house_value(features, model_name, predictor_to_use)
5 start_time = time.time()
6
----> 7 response = predictor_to_use.predict(features, target_model=model_name)
8
9 response_json = json.loads(response)

~/anaconda3/envs/python3/lib/python3.6/site-packages/sagemaker/predictor.py in predict(self, data, initial_args, target_model, target_variant)
111
112 request_args = self._create_request_args(data, initial_args, target_model, target_variant)
--> 113 response = self.sagemaker_session.sagemaker_runtime_client.invoke_endpoint(**request_args)
114 return self._handle_response(response)
115

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/client.py in _api_call(self, *args, **kwargs)
314 "%s() only accepts keyword arguments." % py_operation_name)
315 # The "self" in this scope is referring to the BaseClient.
--> 316 return self._make_api_call(operation_name, kwargs)
317
318 _api_call.__name__ = str(py_operation_name)

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/client.py in _make_api_call(self, operation_name, api_params)
620 else:
621 http, parsed_response = self._make_request(
--> 622 operation_model, request_dict, request_context)
623
624 self.meta.events.emit(

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/client.py in _make_request(self, operation_model, request_dict, request_context)
639 def _make_request(self, operation_model, request_dict, request_context):
640 try:
--> 641 return self._endpoint.make_request(operation_model, request_dict)
642 except Exception as e:
643 self.meta.events.emit(

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/endpoint.py in make_request(self, operation_model, request_dict)
100 logger.debug("Making request for %s with params: %s",
101 operation_model, request_dict)
--> 102 return self._send_request(request_dict, operation_model)
103
104 def create_request(self, params, operation_model=None):

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/endpoint.py in _send_request(self, request_dict, operation_model)
135 request, operation_model, context)
136 while self._needs_retry(attempts, operation_model, request_dict,
--> 137 success_response, exception):
138 attempts += 1
139 # If there is a stream associated with the request, we need

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/endpoint.py in _needs_retry(self, attempts, operation_model, request_dict, response, caught_exception)
254 event_name, response=response, endpoint=self,
255 operation=operation_model, attempts=attempts,
--> 256 caught_exception=caught_exception, request_dict=request_dict)
257 handler_response = first_non_none_response(responses)
258 if handler_response is None:

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/hooks.py in emit(self, event_name, **kwargs)
354 def emit(self, event_name, **kwargs):
355 aliased_event_name = self._alias_event_name(event_name)
--> 356 return self._emitter.emit(aliased_event_name, **kwargs)
357
358 def emit_until_response(self, event_name, **kwargs):

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/hooks.py in emit(self, event_name, **kwargs)
226 handlers.
227 """
--> 228 return self._emit(event_name, kwargs)
229
230 def emit_until_response(self, event_name, **kwargs):

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/hooks.py in _emit(self, event_name, kwargs, stop_on_response)
209 for handler in handlers_to_call:
210 logger.debug('Event %s: calling handler %s', event_name, handler)
--> 211 response = handler(**kwargs)
212 responses.append((handler, response))
213 if stop_on_response and response is not None:

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/retryhandler.py in __call__(self, attempts, response, caught_exception, **kwargs)
181
182 """
--> 183 if self._checker(attempts, response, caught_exception):
184 result = self._action(attempts=attempts)
185 logger.debug("Retry needed, action of: %s", result)

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/retryhandler.py in __call__(self, attempt_number, response, caught_exception)
249 def __call__(self, attempt_number, response, caught_exception):
250 should_retry = self._should_retry(attempt_number, response,
--> 251 caught_exception)
252 if should_retry:
253 if attempt_number >= self._max_attempts:

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/retryhandler.py in _should_retry(self, attempt_number, response, caught_exception)
275 # If we've exceeded the max attempts we just let the exception
276 # propogate if one has occurred.
--> 277 return self._checker(attempt_number, response, caught_exception)
278
279

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/retryhandler.py in __call__(self, attempt_number, response, caught_exception)
315 for checker in self._checkers:
316 checker_response = checker(attempt_number, response,
--> 317 caught_exception)
318 if checker_response:
319 return checker_response

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/retryhandler.py in __call__(self, attempt_number, response, caught_exception)
221 elif caught_exception is not None:
222 return self._check_caught_exception(
--> 223 attempt_number, caught_exception)
224 else:
225 raise ValueError("Both response and caught_exception are None.")

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/retryhandler.py in _check_caught_exception(self, attempt_number, caught_exception)
357 # the MaxAttemptsDecorator is not interested in retrying the exception
358 # then this exception just propogates out past the retry code.
--> 359 raise caught_exception

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/endpoint.py in _do_get_response(self, request, operation_model)
198 http_response = first_non_none_response(responses)
199 if http_response is None:
--> 200 http_response = self._send(request)
201 except HTTPClientError as e:
202 return (None, e)

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/endpoint.py in _send(self, request)
267
268 def _send(self, request):
--> 269 return self.http_session.send(request)
270
271

~/anaconda3/envs/python3/lib/python3.6/site-packages/botocore/httpsession.py in send(self, request)
287 raise ConnectTimeoutError(endpoint_url=request.url, error=e)
288 except URLLib3ReadTimeoutError as e:
--> 289 raise ReadTimeoutError(endpoint_url=request.url, error=e)
290 except ProtocolError as e:
291 raise ConnectionClosedError(

ReadTimeoutError: Read timeout on endpoint URL: "https://runtime.sagemaker.us-west-2.amazonaws.com/endpoints/inference-pipeline-ep-2020-08-21-18-37-51/invocations"
```

Start with advanced_functionality/multi_model_linear_learner_home_value/linear_learner_multi_model_endpoint_inf_pipeline.ipynb and inspect invoke_multiple_models_mme, predict_one_house_value, and the predictor call shown in the traceback. Run the cell invoking the first 100 models and determine the expected behavior for the ReadTimeoutError. Done means the documented invocation completes without the reported timeout or clearly reports the failure.

Written by the indexing model from the issue text.