Open-EO / Open-EO/FuseTS

Automatically upload dependencies to artifactory after build

Open
#94 0 comments 0 reactions 1 assignee View on GitHub

@JanssenBrm is already working on this.

Since Dec 1, 2023.

enhancement T4.1 Define algo enhancement WP5
Dominant language
Jupyter Notebook
Stars
30
Forks
8
PR merge metrics
No merged PRs in 30d

Description

The FuseTS services are reliant on a specific set of dependencies (such as vam.whittaker, GPy, etc.) and its own code. Consequently, it is essential for these dependencies to be installed within the execution environment. While accomplishing this for local execution is relatively straightforward, it presents greater challenges within the context of openEO.

In the current version of the library, the dependencies are provided through openEO's job_options, as shown below:

mogpr_job = mogpr.execute_batch('./mogpr.json', out_format="json", title=f'FuseTS - MOGPR', job_options={
    'udf-dependency-archives': [
        'https://artifactory.vgt.vito.be:443/auxdata-public/ai4food/fusets_venv.zip#tmp/venv',
        'https://artifactory.vgt.vito.be:443/auxdata-public/ai4food/fusets.zip#tmp/venv_static'
    ],
    'executor-memory': '7g'
})

Even though this works as is, it does require an update of the archives each time a change is done in the FuseTS library. A possible improvement would be to either:

  • Find an alternative solution to providing the dependencies in openEO that can be supported on any openEO backend
  • Include the creation and upload of the archives part of the build process

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.