[Bug]: Dimension Lookup from Cache failing: ClassCastException
- Dominant language
- Java
- Stars
- 1.5k
- Forks
- 476
- Avg merge
- 18h 32m
- Merged PRs (30d)
- 216
Description
### Apache Hop version?
2.10.0
### Java version?
21.0.4
### Operating system
Windows
### What happened?
Fill a dimension table with timestamp column as date_from and date_to.
Use 'Dimension lookup/update' to lookup the technical key, activate caching and preloading of cache.
During lookup, class cast exception occurs:
` ERROR: Because of an error this transform can't continue:
2024/11/14 09:39:16 - lookup material_key.0 - java.lang.RuntimeException: java.lang.ClassCastException: class java.util.Date cannot be cast to class java.sql.Timestamp (java.util.Date is in module java.base of loader 'bootstrap'; java.sql.Timestamp is in module java.sql of loader 'platform')
2024/11/14 09:39:16 - lookup material_key.0 - java.lang.ClassCastException: class java.util.Date cannot be cast to class java.sql.Timestamp (java.util.Date is in module java.base of loader 'bootstrap'; java.sql.Timestamp is in module java.sql of loader 'platform')
2024/11/14 09:39:16 - lookup material_key.0 - ERROR: org.apache.hop.core.exception.HopException:
2024/11/14 09:39:16 - lookup material_key.0 - java.lang.RuntimeException: java.lang.ClassCastException: class java.util.Date cannot be cast to class java.sql.Timestamp (java.util.Date is in module java.base of loader 'bootstrap'; java.sql.Timestamp is in module java.sql of loader 'platform')
2024/11/14 09:39:16 - lookup material_key.0 - java.lang.ClassCastException: class java.util.Date cannot be cast to class java.sql.Timestamp (java.util.Date is in module java.base of loader 'bootstrap'; java.sql.Timestamp is in module java.sql of loader 'platform')
2024/11/14 09:39:16 - lookup material_key.0 -
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.pipeline.transforms.dimensionlookup.DimensionCache.lookupRow(DimensionCache.java:172)
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.pipeline.transforms.dimensionlookup.DimensionLookup.lookupValues(DimensionLookup.java:421)
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.pipeline.transforms.dimensionlookup.DimensionLookup.processRow(DimensionLookup.java:212)
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.pipeline.transform.RunThread.run(RunThread.java:54)
2024/11/14 09:39:16 - lookup material_key.0 - at java.base/java.lang.Thread.run(Thread.java:1583)
2024/11/14 09:39:16 - lookup material_key.0 - Caused by: java.lang.RuntimeException: java.lang.ClassCastException: class java.util.Date cannot be cast to class java.sql.Timestamp (java.util.Date is in module java.base of loader 'bootstrap'; java.sql.Timestamp is in module java.sql of loader 'platform')
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.pipeline.transforms.dimensionlookup.DimensionCache.compare(DimensionCache.java:230)
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.pipeline.transforms.dimensionlookup.DimensionCache.compare(DimensionCache.java:36)
2024/11/14 09:39:16 - lookup material_key.0 - at java.base/java.util.Collections.indexedBinarySearch(Collections.java:335)
2024/11/14 09:39:16 - lookup material_key.0 - at java.base/java.util.Collections.binarySearch(Collections.java:323)
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.pipeline.transforms.dimensionlookup.DimensionCache.lookupRow(DimensionCache.java:99)
2024/11/14 09:39:16 - lookup material_key.0 - ... 4 more
2024/11/14 09:39:16 - lookup material_key.0 - Caused by: java.lang.ClassCastException: class java.util.Date cannot be cast to class java.sql.Timestamp (java.util.Date is in module java.base of loader 'bootstrap'; java.sql.Timestamp is in module java.sql of loader 'platform')
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.core.row.value.ValueMetaTimestamp.getTimestamp(ValueMetaTimestamp.java:125)
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.core.row.value.ValueMetaTimestamp.getDate(ValueMetaTimestamp.java:72)
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.pipeline.transforms.dimensionlookup.DimensionCache.compare(DimensionCache.java:204)
2024/11/14 09:39:16 - lookup material_key.0 - ... 8 more`
### Issue Priority
Priority: 2
### Issue Component
Component: Transforms
Contributor guide
Research direction
Start with pipeline/transforms/dimensionlookup/DimensionCache.java, especially lookupRow and compare, then inspect ValueMetaTimestamp.getTimestamp and getDate. Reproduce a Dimension lookup/update using timestamp date_from and date_to fields with caching and cache preloading enabled. Done means the lookup completes without the reported ClassCastException.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100