[Feature Request]: Add reusable metadata for shared functions and code across pipelines and workflows
- Dominant language
- Java
- Stars
- 1.5k
- Forks
- 476
- Avg merge
- 18h 32m
- Merged PRs (30d)
- 216
Description
### What would you like to happen?
Description:
Currently, when the same logic is needed in multiple pipelines or workflows, it often has to be duplicated in JavaScript scripts, Java classes, or other embedded code snippets inside components. This creates repetition, increases maintenance cost, and makes standardization harder.
The proposal is to create a new metadata type to store and manage reusable functions, routines, or code snippets, allowing them to be centrally referenced by different pipelines and workflows.
Objective:
Avoid code duplication and improve maintenance, reuse, and governance of shared logic.
Expected behavior:
allow defining reusable functions or routines as metadata;
allow this metadata to be referenced from multiple pipelines and workflows;
support different implementation types, such as JavaScript, script, and Java classes;
enable centralized maintenance, so that changes in the metadata are reflected in all flows that use it;
reduce the need to copy and paste code across different transforms.
Use cases:
business rules reused across multiple flows;
helper functions for field processing;
standardized validations;
value conversion logic;
common logic used in scripts or custom classes.
Benefits:
elimination of duplicated code;
centralized maintenance;
greater consistency across pipelines;
improved technical governance;
easier evolution and correction of shared rules.
Possible future extensions:
function versioning;
dependency management;
standardized input and output parameters;
embedded metadata documentation;
testing and validation for reusable functions.
Note:
The goal is not only to store code, but to let Hop treat this logic as a reusable and manageable resource, decoupling implementation from the pipeline or workflow definition.
### Issue Priority
Priority: 3
### Issue Component
Component: Metadata
Contributor guide
Research direction
The issue names no files, tests, or entry points. Start by reviewing the existing Metadata component and how pipelines and workflows reference metadata; clarify the reusable code types, parameters, lifecycle, and integration boundaries. Done should include centrally managed reusable logic that multiple flows can reference and that supports the listed implementation types.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, javascript
- Domain
- backend, data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100