[Feature Request]: The ability to set rowset size per pipeline
- Dominant language
- Java
- Stars
- 1.5k
- Forks
- 476
- Avg merge
- 19h
- Merged PRs (30d)
- 227
Description
### What would you like to happen?
In PDI we were able to set the rowset size on a per transformation. There were scenarios where this was useful. In Hop it a global value per pipeline runtime configuration. Would it be possible to add this functionality to override this value on a per pipeline basis like PDI?
### Issue Priority
Priority: 3
### Issue Component
Component: Metadata
Contributor guide
Research direction
No files, tests, or entry points are named. Start by tracing how Hop reads the global rowset size from pipeline runtime configuration and how pipeline metadata is stored, then compare that behavior with the per-transformation setting described from PDI. Done means a pipeline-specific value can override the global value without affecting other pipelines.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100