kestra-io / kestra-io/plugin-graalvm
Control columns order for FileTransform tasks
- Dominant language
- Java
- Stars
- 0
- Forks
- 1
- Avg merge
- 5h 23m
- Merged PRs (30d)
- 4
Description
### Feature description
Sometimes it is useful to define a strict order of column outputted by FileTransform task (jython in my case). For example, when I want to send data to Batch task for storage or when I need to produce a specific report (CSV, etc.) Currently, all I can do is try to adapt following tasks inputs which is fragile 'cause I can not predict columns order of previous task beforehand and this behaviour is not documented/guaranteed. E.g. if I add some new column to FileTransform the whole order can change.
It would be really nice to be able to specify column order explicitly. Either via some additional task properties or maybe using some intermediate new task that interacts with ION data.
Context: https://kestra-io.slack.com/archives/C03FQKXRK3K/p1720467171927799
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by locating the FileTransform task and reviewing how its task properties and output columns are handled. Decide how an explicit column-order option should be represented, then verify that the configured order is preserved for downstream Batch storage and CSV-style reports and that the behavior is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100