[Bug]: CSV File Input - unable to set constant as null value
- Dominant language
- Java
- Stars
- 1.5k
- Forks
- 476
- Avg merge
- 18h 32m
- Merged PRs (30d)
- 216
Description
### Apache Hop version?
2.9.0
### Java version?
openjdk version "21.0.2" 2024-01-16
### Operating system
Linux
### What happened?
In Hop GUI the CSV file input transform has no place to set whether a constant (such as "NULL") should be parsed as a null value (instead of a "NULL" string).
In the corresponding Metadata Injection view, there is a row labeled `Input fields -> Field null string`, but it seems ignored by Hop.
A partial workaround of the issue (does the job but still leaves unwanted traces in the logs, see #3893 for an example) is:
- setup CSV file input with `Lazy conversion` off, and to read all values as String
- link to a Null If transform, set the fields you need to evaluate and the constants to be parsed as null
- link to a Select values transform, in the Meta-data section specify the fields and their actual type and format (Integer, Date, etc.)
### Issue Priority
Priority: 3
### Issue Component
Component: Hop Gui, Component: Transforms
Contributor guide
Research direction
Start in the Hop GUI CSV file input transform and its Metadata Injection view, reproducing the case where a constant such as "NULL" should become a null value. Check why the `Input fields -> Field null string` setting is ignored; done means the configured constant is parsed as null without requiring the workaround transforms.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100