galaxyproject / galaxyproject/pulsar

Document local relay-based setup; fix single-core default and shared-password auth

Open
#452 2 comments 0 reactions 4 assignees Claimed by @dannon View on GitHub
Dominant language
Python
Stars
46
Forks
62
Avg merge
3d 9h
Merged PRs (30d)
15

Description

## Setup on this machine (Ubuntu 24.04)

I run a Pulsar endpoint that picks up jobs from `test.galaxyproject.org` through the pulsar-relay at `relay.usegalaxy.org`. The steps a new user has to follow:

1. Clone Pulsar, create a venv, `pip install -e .` (or `pip install pulsar-app`)
2. Install Docker engine, add the user to the `docker` group, log out and back in
3. Write `app.yml` with the relay URL, manager name, and a shared password issued by a Galaxy admin
4. Run `pulsar-main -c .`

My `app.yml` (password redacted):

```yaml
conda_auto_init: false
conda_auto_install: false
message_queue_url: "https://relay.usegalaxy.org"
message_queue_username: anton
message_queue_password:
managers:
anton:
staging_directory: /media/anton/data/staging
type: queued_python
```

Once running, jobs from `test.galaxyproject.org` arrive over the relay, stage inputs via HTTPS, and run inside Docker. This works — log excerpt for one job:

```
INFO Detected relay connection string, binding to pulsar-relay at https://relay.usegalaxy.org
INFO Starting relay consumer for topics: ['job_setup_anton', 'job_status_request_anton', 'job_kill_anton']
INFO Successfully authenticated with pulsar-relay, token expires in 3600 seconds
INFO Processing setup message for job_id 1525331
DEBUG Staging input '...' to /media/anton/data/staging/1525331/inputs/...
```

## Problems

### 1. Single core

`type: queued_python` runs one job at a time. The relay docs and the `app.yml` sample don't show how to raise concurrency for a relay-bound manager. A laptop or workstation joining as a Pulsar endpoint should pick up all cores by default, or the docs should show the right knob next to the relay example.

### 2. Auth is a shared password in plaintext

`message_queue_password` lives in plaintext in `app.yml`. Onboarding requires a Galaxy admin to mint a username and password and hand it over out of band. No rotation, no scoping, no self-service revoke.

### 3. No user-friendly auth flow

Claude Code (and `gh auth login`, and most modern CLIs) authenticate this way: the CLI opens a browser, the user clicks approve on the server's UI, and the CLI stores a refreshable token. Pulsar should do the same:

- `pulsar-main login --server https://test.galaxyproject.org` opens the browser
- User approves the endpoint under their Galaxy account
- Pulsar writes a token to a separate config file, not `app.yml`
- Token refreshes on its own; admin or user can revoke from the Galaxy UI

This drops the manual ticket step and gets the secret out of the config file.

## Ask

- Document the relay-based local setup end to end (Docker prereqs, `app.yml` minimum, multi-core)
- Default to multi-core managers, or surface the setting in the relay example
- Browser-based endpoint enrollment, with revocable tokens

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.