developmentseed / developmentseed/openeo-studio

Support UDP API as the Save/Load mechanism for openEO Studio workspaces

Open
#57 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
TypeScript
Stars
8
Forks
0
Avg merge
1d 22h
Merged PRs (30d)
2

Description

### Summary

openEO Studio should use the openEO **User-Defined Processes (UDP) API endpoints** as its primary persistence layer. When a user composes a workflow in Studio, saving it stores the process graph as a UDP on the connected openEO backend. Loading a workspace retrieves a UDP and restores it in the visual editor. This avoids introducing a separate storage layer — the openEO API itself acts as the save/load interface.

Additionally, Studio should allow users to **discover public and shared UDPs** from other users or from the APEx Algorithm Catalogue, load them into the editor, modify them, and save their own versions.

## Motivation

Currently, openEO Studio has no persistence mechanism — workspaces are lost when the session ends. Using the openEO UDP API as the save/load layer is a natural fit: it requires no additional storage infrastructure, and every saved workspace becomes a first-class openEO resource that can be executed, shared, or modified through any openEO client.

The design creates a natural workflow: users prototype in Studio, save as UDPs, share or publish for reuse. Other users can discover existing public UDPs, load them into Studio, adapt them, and save modified versions — building on community work rather than starting from scratch.

Upcoming projects will cover the implementation of this feature.

## Backend support: titiler-openeo

Full UDP lifecycle support is **already implemented** in titiler-openeo as of [PR #158](https://github.com/sentinel-hub/titiler-openeo/pull/158) (merged 10/12/2025), covering all CRUD endpoints and validation. Other openEO backends such as VITO's GeoPySpark-based backend also support the UDP API.

Related closed issues in titiler-openeo: [#126](https://github.com/sentinel-hub/titiler-openeo/issues/126), [#128](https://github.com/sentinel-hub/titiler-openeo/issues/128), [#129](https://github.com/sentinel-hub/titiler-openeo/issues/129), [#130](https://github.com/sentinel-hub/titiler-openeo/issues/130), [#131](https://github.com/sentinel-hub/titiler-openeo/issues/131), [#132](https://github.com/sentinel-hub/titiler-openeo/issues/132), [#137](https://github.com/sentinel-hub/titiler-openeo/issues/137).

## Background: openEO UDP API

### What is a UDP?

A User-Defined Process (UDP) is a reusable, parameterised process graph stored on an openEO backend. It is essentially a saved workflow with:

- A unique **`id`** (letters, numbers, underscores only, unique per user)
- A **`process_graph`** — the chain of openEO process nodes
- **`parameters`** — declared input parameters with JSON Schema types (e.g. temporal extent, geometry, band names)
- Optional **metadata**: `summary`, `description`, `links`

UDPs are distinct from User-Defined Functions (UDFs). UDFs are custom code (Python/R) executed server-side. UDPs are compositions of existing openEO processes stored as reusable building blocks.

### API Endpoints (openEO API v1.2+)

The openEO API defines the following CRUD endpoints under the `User-Defined Processes` section:

| Method | Endpoint | Description |
|--------|----------|-------------|
| `GET` | `/process_graphs` | List all UDPs for the authenticated user |
| `GET` | `/process_graphs/{process_graph_id}` | Get full metadata for a specific UDP |
| `PUT` | `/process_graphs/{process_graph_id}` | Store (create or replace) a UDP |
| `DELETE` | `/process_graphs/{process_graph_id}` | Delete a UDP |
| `POST` | `/validation` | Validate a process graph without storing it |

All endpoints except `/validation` require authentication via Bearer token.

### UDP JSON Structure

A stored UDP follows this schema:

```json
{
"id": "my_ndvi_workflow",
"summary": "Compute NDVI for a given area and time range",
"description": "Calculates the Normalized Difference Vegetation Index...",
"parameters": [
{
"name": "temporal_extent",
"description": "Date range for analysis",
"schema": {
"type": "array",
"subtype": "temporal-interval"
},
"default": ["2024-06-01", "2024-06-30"]
},
{
"name": "geometry",
"description": "Area of interest as GeoJSON",
"schema": {
"type": "object",
"subtype": "geojson"
}
}
],
"process_graph": {
"load1": {
"process_id": "load_collection",
"arguments": {
"id": "SENTINEL2_L2A",
"temporal_extent": {"from_parameter": "temporal_extent"},
"bands": ["B04", "B08"]
}
},
"ndvi1": {
"process_id": "ndvi",
"arguments": {
"data": {"from_node": "load1"},
"nir": "B08",
"red": "B04"
},
"result": true
}
},
"links": []
}
```

### Public/shared UDPs

Some backends (notably VITO's) support a `public` flag when saving a UDP. Public UDPs are accessible to other users via a canonical URL:

```
https://openeo.example/openeo/1.0/processes/u:/
```

Other users can reference public UDPs in their own process graphs using the `namespace` property:

```json
{
"process_id": "my_ndvi_workflow",
"namespace": "https://openeo.example/openeo/1.0/processes/u:johndoe/my_ndvi_workflow",
"arguments": { "temporal_extent": ["2024-01-01", "2024-03-01"] }
}
```

### Key resources

- **openEO API spec (v1.3.0)**: https://api.openeo.org/ — see "User-Defined Processes" section
- **openEO API GitHub**: https://github.com/Open-EO/openeo-api
- **Python client UDP docs**: https://open-eo.github.io/openeo-python-client/udp.html
- **UDP sharing/public access**: https://open-eo.github.io/openeo-python-client/cookbook/udp_sharing.html
- **CDSE UDP tutorial**: https://documentation.dataspace.copernicus.eu/notebook-samples/openeo/UDP.html
- **APEx UDP writer guide**: https://esa-apex.github.io/apex_documentation/guides/udp_writer_guide.html
- **titiler-openeo UDP implementation**: https://github.com/sentinel-hub/titiler-openeo/pull/158
- **DevSeed openeo-udp collection**: https://github.com/developmentseed/openeo-udp

## Scope of work for openEO Studio

### 1. Save workspace as UDP

- When user clicks "Save", Studio serialises the current visual workspace into an openEO process graph
- Studio calls `PUT /process_graphs/{id}` on the connected backend
- User provides a process ID (with validation: lowercase letters, numbers, underscores only)
- Studio extracts parameterisable values (temporal extent, spatial extent, collection, bands) and defines them as UDP `parameters` with appropriate JSON Schema types
- Optional: user can set `summary`, `description`, and `public` flag
- On success, Studio confirms save and stores the UDP ID locally for quick re-save

### 2. Load workspace from UDP

- Studio calls `GET /process_graphs` to list the user's saved UDPs
- User selects a UDP to load
- Studio calls `GET /process_graphs/{id}` to retrieve the full process graph and parameters

**Open design question: restoring the code editor state**

The UDP API stores a process graph (JSON), but Studio users author workflows in a Python code editor. Converting a process graph back into readable, editable Python code is non-trivial, and the reconstructed code will likely differ from what the user originally wrote. Two options are under consideration:

**Option A: Embed the original Python source in the UDP metadata**

When saving, Studio stores the user's original Python code inside the UDP `description` field, wrapped in a dedicated markdown code block (e.g. ` ```openeo-studio-source ... ``` `). On load, Studio extracts this block and restores the code editor exactly as the user left it. The process graph itself remains the authoritative executable representation.

| Pros | Cons |
|------|------|
| Preserves the user's original code, variable names, comments, formatting | Relies on a convention in the `description` field — other clients may overwrite or strip it |
| Perfect round-trip fidelity for Studio users | The Python source and the process graph could drift out of sync if the UDP is edited outside Studio |
| Simple to implement on the Studio side | Uses the `description` field for a purpose beyond its original intent |

**Option B: Reconstruct Python code from the process graph**

On load, Studio deserialises the process graph JSON and generates Python code using the same logic as the "Export as Source Code" feature in the openEO web editor. No original source is stored — the code is always derived from the graph.

| Pros | Cons |
|------|------|
| The process graph is the single source of truth — no sync issues | The generated code is less readable than hand-written code (auto-generated variable names, no comments) |
| Works for any UDP, including those created outside Studio | Users lose their original code style and structure |
| No dependency on metadata conventions | May be confusing for users who expect to see their original code |

**Option C (hybrid): Store source in metadata, fall back to reconstruction**

Studio attempts to extract the original source from the UDP metadata. If not found (e.g. the UDP was created via Python client or another tool), it falls back to generating code from the process graph. This gives the best experience for Studio-originated UDPs while remaining compatible with UDPs from any source.

A decision on the preferred approach is needed before implementation.

### 3. Discover public/shared UDPs

- Studio provides a browse/search interface for public UDPs
- Sources: the authenticated user's own UDPs, public UDPs from other users, UDPs registered in the APEx Algorithm Catalogue
- Users can inspect UDP parameters, description, and documentation before loading
- Loading a public UDP into Studio for execution or further modification

### 4. Validate before save/execute

- Before saving or executing, Studio can call `POST /validation` to check the process graph
- Validation errors are displayed inline in the visual editor with guidance on how to fix them

## Implementation notes

This issue captures the feature request and design considerations. It must be properly reviewed and designed before execution in the context of a future project. The scope, approach (particularly the deserialisation strategy), and technical details should be refined during project kick-off.

- Authentication is required for all UDP CRUD operations — Studio must pass the Bearer token from the user's openEO session
- The `PUT` endpoint is idempotent (create or replace), so "Save" and "Save As" can use the same endpoint with different IDs
- Process graph serialisation/deserialisation must handle `from_parameter` and `from_node` references correctly to round-trip between visual editor and UDP JSON
- Studio should work with any openEO backend that implements the UDP endpoints, not just a specific backend

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.