[BUG] Jupyter notebook execution behaving randomly

@mhamilton723 is already working on this.

Since Jul 10, 2023.

• #2284 by @memoryz — merged

### SynapseML version

using docker installation mentioned here https://microsoft.github.io/SynapseML/docs/getting_started/installation/#docker

### System information

- **Language version** (e.g. python 3.8, scala 2.12):
- **Spark Version** (e.g. 3.2.3):
- **Spark Platform** (e.g. Synapse, Databricks):

using the docker setup mentioened here: https://microsoft.github.io/SynapseML/docs/getting_started/installation/#docker

Note: using Macbook Pro (Apple M1 Pro chip)

### Describe the problem

sometimes the same code runs sometimes it takes too much time(i feel it hangs execution), attaching screenshot of same:

Although it stucks in different part of notebook if separate attempts. even simple commands like echo fails.

attaching logs as well in below section: (Note: i had to restart jupyter kernal many time to start execution from begining, it randomaly stops further execution. i had to start jupyter in debug mode to check)

### Code to reproduce issue

NA

### Other info / logs

demo-notebook-1 | [D 15:06:15.332 NotebookApp] 304 GET /static/components/MathJax/MathJax.js?config=TeX-AMS-MML_HTMLorMML-full,Safe&delayStartupUntil=configured (172.20.0.1) 118.460000ms
demo-notebook-1 | [D 15:06:15.359 NotebookApp] 304 GET /custom/custom.css (172.20.0.1) 89.880000ms
demo-notebook-1 | [D 15:06:15.375 NotebookApp] 304 GET /static/components/MathJax/MathJax.js?config=TeX-AMS-MML_HTMLorMML-full,Safe&delayStartupUntil=configured (172.20.0.1) 8.050000ms
demo-notebook-1 | [D 15:06:15.796 NotebookApp] 304 GET /static/components/MathJax/config/TeX-AMS-MML_HTMLorMML-full.js?V=2.7.9 (172.20.0.1) 16.000000ms
demo-notebook-1 | [D 15:06:15.956 NotebookApp] 304 GET /custom/custom-preload.js?v=20230706121415 (172.20.0.1) 18.320000ms
demo-notebook-1 | [D 15:06:15.967 NotebookApp] 304 GET /static/components/MathJax/config/TeX-AMS-MML_HTMLorMML-full.js?V=2.7.9 (172.20.0.1) 5.560000ms
demo-notebook-1 | [D 15:06:15.980 NotebookApp] 304 GET /custom/custom-preload.js?v=20230706121415 (172.20.0.1) 5.570000ms
demo-notebook-1 | [D 15:06:16.026 NotebookApp] 304 GET /static/components/MathJax/config/Safe.js?V=2.7.9 (172.20.0.1) 8.610000ms
demo-notebook-1 | [D 15:06:16.255 NotebookApp] Paths used for configuration of notebook:
demo-notebook-1 | /etc/jupyter/nbconfig/notebook.json
demo-notebook-1 | [D 15:06:16.295 NotebookApp] Paths used for configuration of notebook:
demo-notebook-1 | /usr/local/etc/jupyter/nbconfig/notebook.d/widgetsnbextension.json
demo-notebook-1 | /usr/local/etc/jupyter/nbconfig/notebook.json
demo-notebook-1 | [D 15:06:16.313 NotebookApp] Paths used for configuration of notebook:
demo-notebook-1 | /root/.local/etc/jupyter/nbconfig/notebook.json
demo-notebook-1 | [D 15:06:16.332 NotebookApp] Paths used for configuration of notebook:
demo-notebook-1 | /root/.jupyter/nbconfig/notebook.json
demo-notebook-1 | [D 15:06:16.349 NotebookApp] 200 GET /api/config/notebook?_=1688655975811 (172.20.0.1) 298.760000ms
demo-notebook-1 | [D 15:06:16.361 NotebookApp] Paths used for configuration of common:
demo-notebook-1 | /etc/jupyter/nbconfig/common.json
demo-notebook-1 | [D 15:06:16.366 NotebookApp] Paths used for configuration of common:
demo-notebook-1 | /usr/local/etc/jupyter/nbconfig/common.json
demo-notebook-1 | [D 15:06:16.372 NotebookApp] Paths used for configuration of common:
demo-notebook-1 | /root/.local/etc/jupyter/nbconfig/common.json
demo-notebook-1 | [D 15:06:16.379 NotebookApp] Paths used for configuration of common:
demo-notebook-1 | /root/.jupyter/nbconfig/common.json
demo-notebook-1 | [D 15:06:16.385 NotebookApp] 200 GET /api/config/common?_=1688655975812 (172.20.0.1) 31.180000ms
demo-notebook-1 | [D 15:06:16.440 NotebookApp] Found kernel python3 in /usr/local/share/jupyter/kernels
demo-notebook-1 | [D 15:06:16.474 NotebookApp] 304 GET /api/kernelspecs (172.20.0.1) 65.320000ms
demo-notebook-1 | [D 15:06:16.501 NotebookApp] 304 GET /static/components/MathJax/config/Safe.js?V=2.7.9 (172.20.0.1) 86.230000ms
demo-notebook-1 | [D 15:06:16.508 NotebookApp] 304 GET /static/components/MathJax/extensions/TeX/begingroup.js?V=2.7.9 (172.20.0.1) 90.120000ms
demo-notebook-1 | [W 15:06:16.612 NotebookApp] Notebook test_docker_compose/synapses_if.ipynb is not trusted
demo-notebook-1 | [D 15:06:16.636 NotebookApp] 200 GET /api/contents/test_docker_compose/synapses_if.ipynb?type=notebook&_=1688655975813 (172.20.0.1) 151.380000ms
demo-notebook-1 | [D 15:06:16.640 NotebookApp] 304 GET /custom/custom.js?v=20230706121415 (172.20.0.1) 150.970000ms
demo-notebook-1 | [D 15:06:16.643 NotebookApp] 304 GET /static/components/MathJax/extensions/TeX/newcommand.js?V=2.7.9 (172.20.0.1) 145.860000ms
demo-notebook-1 | [D 15:06:16.650 NotebookApp] 304 GET /static/components/MathJax/extensions/MathML/content-mathml.js?V=2.7.9 (172.20.0.1) 137.110000ms
demo-notebook-1 | [D 15:06:16.689 NotebookApp] 304 GET /static/components/MathJax/jax/output/HTML-CSS/fonts/STIX-Web/fontdata.js?V=2.7.9 (172.20.0.1) 19.670000ms
demo-notebook-1 | [D 15:06:16.697 NotebookApp] 304 GET /static/components/MathJax/extensions/TeX/begingroup.js?V=2.7.9 (172.20.0.1) 24.910000ms
demo-notebook-1 | [D 15:06:16.700 NotebookApp] 304 GET /custom/custom.js?v=20230706121415 (172.20.0.1) 25.490000ms
demo-notebook-1 | [D 15:06:16.704 NotebookApp] 304 GET /static/components/MathJax/extensions/Safe.js?V=2.7.9 (172.20.0.1) 25.040000ms
demo-notebook-1 | [D 15:06:16.714 NotebookApp] 304 GET /nbextensions/jupyter-js-widgets/extension.js?v=20230706121415 (172.20.0.1) 22.470000ms
demo-notebook-1 | [D 15:06:16.731 NotebookApp] 304 GET /static/components/MathJax/extensions/TeX/newcommand.js?V=2.7.9 (172.20.0.1) 12.760000ms
demo-notebook-1 | [D 15:06:16.736 NotebookApp] 304 GET /static/components/MathJax/extensions/MathML/content-mathml.js?V=2.7.9 (172.20.0.1) 11.290000ms
demo-notebook-1 | [D 15:06:16.770 NotebookApp] 200 GET /nbextensions/jupyter-js-widgets/extension.js?v=20230706121415 (172.20.0.1) 28.960000ms
demo-notebook-1 | [D 15:06:16.793 NotebookApp] 200 GET /static/components/MathJax/extensions/Safe.js?V=2.7.9 (172.20.0.1) 48.530000ms
demo-notebook-1 | [D 15:06:16.800 NotebookApp] 304 GET /static/components/MathJax/jax/output/HTML-CSS/fonts/STIX-Web/fontdata.js?V=2.7.9 (172.20.0.1) 51.410000ms
demo-notebook-1 | [D 15:06:16.845 NotebookApp] 200 GET /api/contents/test_docker_compose/synapses_if.ipynb/checkpoints?_=1688655975814 (172.20.0.1) 21.340000ms
demo-notebook-1 | [D 15:06:16.874 NotebookApp] 201 POST /api/sessions (172.20.0.1) 43.300000ms
demo-notebook-1 | [D 15:06:17.003 NotebookApp] Initializing websocket connection /api/kernels/3295e79c-6f1d-4e50-8732-1eedf72d2109/channels
demo-notebook-1 | [D 15:06:17.045 NotebookApp] 101 GET /api/kernels/3295e79c-6f1d-4e50-8732-1eedf72d2109/channels?session_id=277e9fc2dd9c4e3db74ad8e64f847349 (172.20.0.1) 46.530000ms
demo-notebook-1 | [D 15:06:17.047 NotebookApp] Opening websocket /api/kernels/3295e79c-6f1d-4e50-8732-1eedf72d2109/channels
demo-notebook-1 | [D 15:06:17.048 NotebookApp] Getting buffer for 3295e79c-6f1d-4e50-8732-1eedf72d2109
demo-notebook-1 | [D 15:06:17.052 NotebookApp] Connecting to: tcp://127.0.0.1:42545
demo-notebook-1 | [D 15:06:17.079 NotebookApp] Connecting to: tcp://127.0.0.1:46619
demo-notebook-1 | [D 15:06:17.084 NotebookApp] Connecting to: tcp://127.0.0.1:39049
demo-notebook-1 | [D 15:06:17.088 NotebookApp] Connecting to: tcp://127.0.0.1:36609
demo-notebook-1 | [D 15:06:17.092 NotebookApp] Nudge: not nudging busy kernel 3295e79c-6f1d-4e50-8732-1eedf72d2109
demo-notebook-1 | [D 15:06:17.984 NotebookApp] 304 GET /static/base/images/favicon-busy-1.ico (172.20.0.1) 3.630000ms
demo-notebook-1 | [D 15:08:17.074 NotebookApp] 200 GET /api/contents/test_docker_compose/synapses_if.ipynb?content=0&_=1688655975815 (172.20.0.1) 61.970000ms
demo-notebook-1 | [I 15:08:17.113 NotebookApp] Saving file at /test_docker_compose/synapses_if.ipynb
demo-notebook-1 | [D 15:08:17.115 NotebookApp] Saving /notebooks/features/test_docker_compose/synapses_if.ipynb
demo-notebook-1 | [W 15:08:17.123 NotebookApp] Notebook test_docker_compose/synapses_if.ipynb is not trusted
demo-notebook-1 | [D 15:08:17.286 NotebookApp] 200 PUT /api/contents/test_docker_compose/synapses_if.ipynb (172.20.0.1) 185.620000m

### What component(s) does this bug affect?

- [ ] `area/cognitive`: Cognitive project
- [X] `area/core`: Core project
- [X] `area/deep-learning`: DeepLearning project
- [ ] `area/lightgbm`: Lightgbm project
- [ ] `area/opencv`: Opencv project
- [ ] `area/vw`: VW project
- [ ] `area/website`: Website
- [X] `area/build`: Project build system
- [ ] `area/notebooks`: Samples under notebooks folder
- [X] `area/docker`: Docker usage
- [ ] `area/models`: models related issue

### What language(s) does this bug affect?

- [ ] `language/scala`: Scala source code
- [ ] `language/python`: Pyspark APIs
- [ ] `language/r`: R APIs
- [ ] `language/csharp`: .NET APIs
- [ ] `language/new`: Proposals for new client languages

### What integration(s) does this bug affect?

- [ ] `integrations/synapse`: Azure Synapse integrations
- [ ] `integrations/azureml`: Azure ML integrations
- [ ] `integrations/databricks`: Databricks integrations