Azure / Azure/azure-sdk-for-python

`JupyterLabJobService` endpoint returns “User <UPN> does not have access to node <node-id>” on running AML job

Open
#45,108 2 comments 0 reactions 0 assignees View on GitHub
customer-reported Machine Learning needs-team-attention question Service Attention
Dominant language
Python
Stars
5.6k
Forks
3.4k
Avg merge
1d 21h
Merged PRs (30d)
193

Description

- **Package Name**:
- azure-ai-ml
- **Package Version**:
- 1.25.0
- **Operating System**:
- Windows 11 Enterprise (24H2) x86_64
- **Python Version**:
- 3.12.9

## Describe the bug
When enabling `JupyterLabJobService` for an Azure ML v2 `command` job running on a **compute cluster** (not a compute instance), the job runs successfully and the JupyterLab service endpoint is returned by `ml_client.jobs.show_services(...)` / `az ml job show-services`. However, opening the returned JupyterLab URL in a browser shows an authorization page stating that the user does not have access to the node.

This prevents using `JupyterLabJobService` for interactive debugging (terminal/REPL) on cluster jobs.

## To Reproduce

1. Create an Azure ML v2 `command` job targeting a compute cluster.
2. Enable `JupyterLabJobService` on the job, e.g.:

```{python}
from azure.ai.ml import command
from azure.ai.ml.entities import JupyterLabJobService

job = command(
code="./src",
command="python train.py; sleep infinity",
environment="",
compute="",
services={"jupyter": JupyterLabJobService()},
)
```

3. Submit the job and wait until it reaches `Running`.
4. Retrieve the service endpoint:

* Python: `ml_client.jobs.show_services("", node_index=0)`
* or CLI: `az ml job show-services --name --node-index 0`
* or use the AzureML UI

5. Open the returned JupyterLab URL in a browser or the hyperlink in the UI.

## Expected behavior

The JupyterLab page loads (after authentication) and provides access to a terminal/REPL inside the running job container.

## Actual behavior

Opening the returned JupyterLab endpoint shows an error page indicating that the signed-in user does not have access to the node.

## Screenshots

Image

## Additional context

* The job itself runs successfully; only the interactive endpoint is inaccessible.
* Region: West Europe.
* Compute runs on a VNet

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.