Open-EO / Open-EO/openeo-python-driver

Heavy `/processes` due to `mask_scl_dilation`

Open
#139 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
12
Forks
8
Avg merge
6d 3h
Merged PRs (30d)
4

Description

While checking the effect of some caching improvements in the aggregator (https://github.com/Open-EO/openeo-aggregator/issues/2) I noticed that the web editor is still hanging on the /processes request, mainly due to the relatively large size of the response (> 1MB).

I had a quick look through the data and it turns out that 77% of the response is spent on the mask_scl_dilation process. The second largest one is 100 times smaller:
Screenshot from 2022-09-23 12-07-12

mask_scl_dilation is so heavy because it includes a "process_graph" with a very large apply_kernel, but as far as I understand, that process_graph is actually not used:
https://github.com/Open-EO/openeo-python-driver/blob/69b98f07ff1e6621f2b0e8e5c2448d5e014c7f7e/openeo_driver/ProcessGraphDeserializer.py#L1773-L1786

It's just there for illustration purposes I guess.

can't we just drop the process_graph from https://github.com/Open-EO/openeo-python-driver/blob/master/openeo_driver/specs/openeo-processes/experimental/mask_scl_dilation.json @jdries ?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Inspect openeo_driver/specs/openeo-processes/experimental/mask_scl_dilation.json and the referenced ProcessGraphDeserializer.py lines 1773-1786 to confirm whether the large process_graph is used. Compare the /processes response before and after removing it; done means mask_scl_dilation no longer dominates the response size without affecting process handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.