dlt-hub / dlt-hub/dlt

Dashboard: switching profile keeps the previous profile pipeline selected, causing an attach error

Open
#4,094 0 comments 0 reactions 1 assignee Claimed by @anuunchin View on GitHub
bug needs maintainer
Dominant language
Python
Stars
5.9k
Forks
600
Avg merge
1d 14h
Merged PRs (30d)
38

Description

### dlt version

`1.28.1`

### Describe the problem

In the marimo dashboard, switching the **profile** via the profile dropdown while a
pipeline is selected leaves the previously selected pipeline "stuck", and — because the
pipelines directory is profile-scoped — the dashboard then tries to attach to a pipeline
that does not exist under the new profile and shows a **"Could not attach to pipeline …"**
error, instead of cleanly showing the new profile's pipelines (or its landing page).

Root cause (code trace in `dlt/_workspace/`):

1. The profile dropdown's `on_change` (`helpers/dashboard/dlt_dashboard.py`, `_on_profile_change`)
sets only the `profile` URL query param — it never clears the `pipeline` param.
2. The `pipeline` param therefore retains the pipeline selected under the previous profile.
3. The query-vars cell re-reads it into `mo_query_var_pipeline_name` (stale value).
4. The discovery cell force-adds that name via `list_local_pipelines(additional_pipelines=[…, mo_query_var_pipeline_name])`
(`cli/utils.py`), and defaults the pipeline selector value to `[mo_query_var_pipeline_name]`
because it is truthy.
5. The pipelines directory is profile-scoped — `get_dlt_pipelines_dir()` →
`run_context.active().get_data_entity("pipelines")`, and the workspace data dir is
`os.path.join(data_dir, profile)` (`_workspace/run_context.py`). The new profile's dir
does not contain the old pipeline.
6. The `home` cell calls `get_pipeline(stale_name, new_profile_dir)`, attach fails, and the
"Could not attach to pipeline …" error callout is shown.

The dropdown is still rendered in the error path, so the state is recoverable by manually
re-selecting — but the default experience after a profile switch is an error rather than
the new profile's content.

### Expected behavior

Switching profiles should not carry the previous profile's pipeline selection over. On a
profile change the dashboard should clear (or validate against the new profile's pipeline
list) the `pipeline` selection/URL param, then apply the normal default (auto-select the
most recent pipeline in the new profile, or show the landing page if the new profile has
no pipelines).

### Steps to reproduce

1. In a profiles-aware workspace with at least two profiles (e.g. `tests`, `prod`), create
a pipeline under each profile (different names), so each profile-scoped pipelines dir
holds a different pipeline.
2. Launch the dashboard and open it on profile A with its pipeline selected
(e.g. `?profile=tests&pipeline=alpha`).
3. Use the **profile dropdown** in the header to switch to profile B (do not navigate via
URL — the URL approach sets both params at once and hides the bug).
4. Observe: the dashboard shows "Could not attach to pipeline alpha" (the profile-A
pipeline), because the stale `pipeline=alpha` param persisted while the pipelines dir
switched to profile B's scope.

### Operating system

macOS (reproducible on all platforms — UI/state logic, not OS-specific)

### Runtime environment

Local

### Python version

3.11

### Additional information

* Confirmed by code trace; the profile-scoped pipelines dir (`_workspace/run_context.py`,
`os.path.join(data_dir, profile)`) is the key factor that turns the stale selection into
an attach error.
* Existing e2e coverage (`tests/e2e/helpers/dashboard/test_e2e.py::test_workspace_profile_*`)
navigates via `?profile=X&pipeline=Y` (both params set together), so it does not exercise
a dropdown-only profile switch and does not catch this.
* Suggested fix direction: in the profile `on_change`, also clear the `pipeline` query
param; and/or in the discovery cell, only honor `mo_query_var_pipeline_name` /
`additional_pipelines` when the name actually exists in the resolved pipelines dir for the
active profile.
* Related (separate) dashboard selector issue: dlt-hub/dlt#4093 (deselecting a pipeline shows the
empty-workspace landing).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.