apache / apache/hop

[Feature Request]: Verify the environment before execution

Open
#6,184 0 comments 0 reactions 0 assignees View on GitHub
awaiting triage Other P3
Dominant language
Java
Stars
1.5k
Forks
476
Avg merge
19h
Merged PRs (30d)
227

Description

### What would you like to happen?

It should be made easy to verify the environment you're in before execution.
Things to verify are for example (but not limited to):

- The disk space available in certain folders
- The total available amount of memory in the JVM
- The amount of available processors
- The availability of certain hosts (ping)
- The presence of (a specific number or amount of) files in certain locations

These things can vary per environment. Obviously it's possible to implement this manually as part of a workflow, but this isn't really encouraging best practices in this regard. Furthermore, certain files might not be needed in development and mandatory in a production environment. Having the ability to have the rules be different per environment is important in this regard.

Architecturally this asks for metadata, in this case "Environment Condition" elements, to be stored somewhere.
If these elements are stored in the project metadata, in a central git repo, there needs to be a match between the name of the environment on the system to know which conditions should be applied.

We can then envision the following:

- An action to verify the environment conditions, an action of the "Projects" plugin. The main option is the name of the "Environment Condition" element to use.
- The ability to define one or more metadata folders per "Project Lifecycle Environment" (under the configuration files) so that the action can find the metadata element.
- When an environment is enabled, the metadata folder needs to be added to the Hop Metadata Providers list. (MultiMetadataProvider)

This way, the metadata can be edited in a standard way in the GUI while being stored separately. The separate metadata can then also be version controlled per environment together with the configuration files.

### Issue Priority

Priority: 3

### Issue Component

Component: Other

Contributor guide

Open the contributing guide

Research direction

No source files or tests are named. Start by reading the Projects plugin, the Project Lifecycle Environment configuration, and MultiMetadataProvider; map how metadata folders and environment names are currently handled. Done means a defined Environment Condition model and an action that verifies the requested disk, JVM memory, processor, host, and file conditions per environment.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
devops
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.