galaxyproject / galaxyproject/pulsar
Document local relay-based setup; fix single-core default and shared-password auth
- Dominant language
- Python
- Stars
- 46
- Forks
- 62
- Avg merge
- 3d 9h
- Merged PRs (30d)
- 15
Description
## Setup on this machine (Ubuntu 24.04)
I run a Pulsar endpoint that picks up jobs from `test.galaxyproject.org` through the pulsar-relay at `relay.usegalaxy.org`. The steps a new user has to follow:
1. Clone Pulsar, create a venv, `pip install -e .` (or `pip install pulsar-app`)
2. Install Docker engine, add the user to the `docker` group, log out and back in
3. Write `app.yml` with the relay URL, manager name, and a shared password issued by a Galaxy admin
4. Run `pulsar-main -c .`
My `app.yml` (password redacted):
```yaml
conda_auto_init: false
conda_auto_install: false
message_queue_url: "https://relay.usegalaxy.org"
message_queue_username: anton
message_queue_password:
managers:
anton:
staging_directory: /media/anton/data/staging
type: queued_python
```
Once running, jobs from `test.galaxyproject.org` arrive over the relay, stage inputs via HTTPS, and run inside Docker. This works — log excerpt for one job:
```
INFO Detected relay connection string, binding to pulsar-relay at https://relay.usegalaxy.org
INFO Starting relay consumer for topics: ['job_setup_anton', 'job_status_request_anton', 'job_kill_anton']
INFO Successfully authenticated with pulsar-relay, token expires in 3600 seconds
INFO Processing setup message for job_id 1525331
DEBUG Staging input '...' to /media/anton/data/staging/1525331/inputs/...
```
## Problems
### 1. Single core
`type: queued_python` runs one job at a time. The relay docs and the `app.yml` sample don't show how to raise concurrency for a relay-bound manager. A laptop or workstation joining as a Pulsar endpoint should pick up all cores by default, or the docs should show the right knob next to the relay example.
### 2. Auth is a shared password in plaintext
`message_queue_password` lives in plaintext in `app.yml`. Onboarding requires a Galaxy admin to mint a username and password and hand it over out of band. No rotation, no scoping, no self-service revoke.
### 3. No user-friendly auth flow
Claude Code (and `gh auth login`, and most modern CLIs) authenticate this way: the CLI opens a browser, the user clicks approve on the server's UI, and the CLI stores a refreshable token. Pulsar should do the same:
- `pulsar-main login --server https://test.galaxyproject.org` opens the browser
- User approves the endpoint under their Galaxy account
- Pulsar writes a token to a separate config file, not `app.yml`
- Token refreshes on its own; admin or user can revoke from the Galaxy UI
This drops the manual ticket step and gets the secret out of the config file.
## Ask
- Document the relay-based local setup end to end (Docker prereqs, `app.yml` minimum, multi-core)
- Default to multi-core managers, or surface the setting in the relay example
- Browser-based endpoint enrollment, with revocable tokens
Contributor guide
Assessment
This issue has not been assessed yet.