galaxyproject / galaxyproject/loom
Seamless local-Pulsar setup for user-defined tools (auto-configure from Galaxy preferences)
- Dominant language
- TypeScript
- Stars
- 14
- Forks
- 12
- Avg merge
- 6d 5h
- Merged PRs (30d)
- 17
Description
Sub-issue of #97. #97 covers the capability and tracks upstream Pulsar blockers (galaxyproject/pulsar#452). This issue covers the **first-run UX** that turns local execution from an expert flow into something a regular Loom user can complete in one sitting.
## Mode of operation
Orbit connects to Galaxy with a user API key (existing flow). The agent uses `galaxy-skills` to register **user-defined tools (UDTs)** in that Galaxy. UDTs are dispatched by Galaxy but execute on a **Pulsar endpoint installed locally on the user's machine**, with tool processes running inside **Docker** on that machine. Inputs and outputs traverse the Galaxy relay (`relay.usegalaxy.org`).
## Prerequisites (admin side)
- Galaxy admin has `user_preferences_extra_conf.yml` enabled with a section exposing relay endpoint URL, manager name, and relay token to users (e.g. `distributed_compute|pulsar_endpoint_url`, `distributed_compute|manager_name`, `distributed_compute|pulsar_token`).
- Galaxy admin has TPV rules routing `tool_type_user_defined` to a per-user destination derived from `user.extra_preferences`. Default TPV behavior **rejects** `tool_type_user_defined`; admin must explicitly accept it on the user-routed destination.
Without both, Loom cannot make UDTs land on local Pulsar — surface this clearly rather than failing silently.
## First-startup flow (what Loom must do)
1. **Galaxy API key prompt** — existing welcome overlay (`app/src/renderer/app.ts:467`) collects Galaxy URL + key.
2. **Fetch user relay info** — call `GET /api/users/current/information/inputs`, parse the extra-preferences section, extract relay URL and manager name. Relay token comes back masked as `__SECRET_PLACEHOLDER__` (see open questions).
3. **Generate local Pulsar config** — new `pulsar_config` component renders `app.yml` and `server.ini` from a template:
```yaml
message_queue_url: https://relay.usegalaxy.org
message_queue_username:
message_queue_password:
managers:
:
type: queued_python
staging_directory:
conda_auto_init: false
conda_auto_install: false
```
(Tools resolve via Docker, so Conda init stays off.)
4. **Install / start Pulsar** — pip-install `pulsar-app` into a managed venv, run `pulsar-main -c ` as a long-lived supervised process.
5. **Verify Docker** — confirm Docker daemon is reachable; block enablement with a clear error if not.
6. **Health surface** — show endpoint status in Orbit (relay auth, last heartbeat, manager name) so the user sees \"local execution is live\" before invoking the first UDT.
## Components to add
- `extensions/loom/pulsar-config.ts` — pure renderer: `{ relayUrl, manager, token, stagingDir } -> { appYml, serverIni }`.
- `app/src/main/pulsar-supervisor.ts` — spawn + monitor local `pulsar-main`, restart on crash, surface status over IPC.
- `extensions/loom/galaxy-api.ts` — add `getUserExtraPreferences()` against `/api/users/current/information/inputs`.
- Preferences UI section: local-Pulsar status, staging dir, \"regenerate config\", \"open logs\".
- Brain awareness: agent should know UDTs route locally and reflect that in plans (\"this step runs on your machine via Docker\").
## Why a separate issue from #97
#97 = broad capability + upstream Pulsar work. This issue = client-side glue that makes setup turn-key once a user has relay credentials. Buildable incrementally today against plaintext-password auth; tightened when galaxyproject/pulsar#452 lands proper enrollment.
## Open questions
- **Relay token retrieval** — `__SECRET_PLACEHOLDER__` blocks API readback. Options: (a) one-time paste on first run, (b) extend Galaxy to return the token to its owner, (c) wait for galaxyproject/pulsar#452 OAuth-style enrollment.
- **Pulsar install shape** — pip-in-managed-venv vs bundled binary vs Pulsar-in-Docker (latter complicates Docker-in-Docker for tool execution).
- **Multi-Galaxy profiles** — one shared local Pulsar with multiple managers, or one Pulsar per profile?
- **Detection of missing TPV setup** — how does Loom detect that the admin destination/rule is absent and warn before the user wastes a job submission?
- **Staging dir default + cleanup** — `~/.loom/pulsar/staging` default? Disk-pressure handling?
- **Reuse of existing Pulsar install** — if the user already has `pulsar-main` running (as in this prototype), can Loom adopt it instead of spawning its own?
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.