Unable to "Create Notebook Job" on Sagemaker Studio when using Custom Image

Hi, I'm assuming Sagemaker Studio uses this package behind the scenes when using the option "Create Notebook Job" (see image below). Please let me know if there is a better repo for this type of issue. This approach is also described on this [AWS blog post](https://aws.amazon.com/blogs/machine-learning/run-notebooks-as-batch-jobs-in-amazon-sagemaker-studio-lab/).


I am trying to use one of our custom images with this option, but I keep getting the error below.

```
+ TRAINING_OUTPUT_PATH=/opt/ml/output/data
+ TRAINING_FAILURE_PATH=/opt/ml/output/failure
+ SM_EXECUTION_INPUT_PATH=/opt/ml/input/data/sagemaker_headless_execution
+ SM_EXECUTION_SYSTEM_PATH=/opt/ml/input/data/sagemaker_headless_execution_system
+ export SM_PAPERMILL_INPUT=/opt/ml/input/data/sagemaker_headless_execution/Untitled10.ipynb
+ SM_PAPERMILL_INPUT=/opt/ml/input/data/sagemaker_headless_execution/Untitled10.ipynb
+ mkdir -p /opt/ml/output/data
+ SM_EXEC_STEP='Prepare the input notebook'
+ '[' -f /opt/ml/input/data/sagemaker_headless_execution/Untitled10.ipynb ']'
+ cp /opt/ml/input/data/sagemaker_headless_execution/Untitled10.ipynb /opt/ml/output/data
+ '[' 1.0 '!=' 1.0 ']'
+ SM_EXEC_STEP='detecting python runtime'
+ '[' '!' -z '' ']'
++ which python
+ PYTHON_EXECUTABLE=/opt/.venv/bin/python
+ echo 'Python executable: /opt/.venv/bin/python'
Python executable: /opt/.venv/bin/python
++ /opt/.venv/bin/python -c 'import sys; version=sys.version_info[:3]; print("{0}.{1}".format(*version))'
+ DEFAULT_PYTHON_VERSION=3.11
+ '[' '!' -z '' ']'
+ SM_EXEC_STEP='creating symbol link to simulate the EFS mounting path'
+ '[' /home/sagemaker-user = /root ']'
++ dirname /home/sagemaker-user
+ mkdir -p /home
+ ln -s /opt/ml/input/data/sagemaker_headless_execution /home/sagemaker-user
+ chown -R 1000 /opt/ml/input/data/sagemaker_headless_execution
+ chgrp -R 100 /opt/ml/input/data/sagemaker_headless_execution
++ id -u
+ [[ 1000 -eq 0 ]]
+ SM_EXEC_STEP='installing the papermill dependency'
++ /opt/.venv/bin/python -c 'import importlib.util; print(None != importlib.util.find_spec("papermill"))'
+ papermill_installed=False
+ '[' False = False ']'
+ /opt/.venv/bin/python -m pip install --no-index --find-links /opt/ml/input/data/sagemaker_headless_execution_system/system/dependencies/3.11 papermill
Looking in links: /opt/ml/input/data/sagemaker_headless_execution_system/system/dependencies/3.11
WARNING: Location '/opt/ml/input/data/sagemaker_headless_execution_system/system/dependencies/3.11' is ignored: it is either a non-existing path or lacks a specific scheme.
ERROR: Could not find a version that satisfies the requirement papermill (from versions: none)
ERROR: No matching distribution found for papermill
+ exit_code=1
+ exit_hook 1
+ set +x
```

I am aware that the image doesn't have papermill on it, so I tried passing the initialization script below:

```
#!/bin/bash
# This script installs a single pip package on a SageMaker Studio Kernel Application

set -eux

# PARAMETERS
PACKAGE=papermill

pip install --upgrade $PACKAGE
```

This results in the same error.

Any ideas on how to address this?

Thanks!

The report names no repository file or test; start by tracing the Create Notebook Job bootstrap shown in the log, especially the Python runtime and papermill installation step, and compare it with the custom-image initialization behavior. Reproduce with the shown Python 3.11 image and verify whether papermill is available at the offline dependency path; done means the job launches successfully with the custom image or the repository documents the unsupported setup.

Written by the indexing model from the issue text.