apache / apache/hop

[Bug]: Dimension Lookup from Cache failing: ClassCastException

Open
#4,562 2 comments 0 reactions 0 assignees View on GitHub
awaiting triage P2 Transforms
Dominant language
Java
Stars
1.5k
Forks
476
Avg merge
18h 32m
Merged PRs (30d)
216

Description

### Apache Hop version?

2.10.0

### Java version?

21.0.4

### Operating system

Windows

### What happened?

Fill a dimension table with timestamp column as date_from and date_to.
Use 'Dimension lookup/update' to lookup the technical key, activate caching and preloading of cache.
During lookup, class cast exception occurs:

` ERROR: Because of an error this transform can't continue:
2024/11/14 09:39:16 - lookup material_key.0 - java.lang.RuntimeException: java.lang.ClassCastException: class java.util.Date cannot be cast to class java.sql.Timestamp (java.util.Date is in module java.base of loader 'bootstrap'; java.sql.Timestamp is in module java.sql of loader 'platform')
2024/11/14 09:39:16 - lookup material_key.0 - java.lang.ClassCastException: class java.util.Date cannot be cast to class java.sql.Timestamp (java.util.Date is in module java.base of loader 'bootstrap'; java.sql.Timestamp is in module java.sql of loader 'platform')
2024/11/14 09:39:16 - lookup material_key.0 - ERROR: org.apache.hop.core.exception.HopException:
2024/11/14 09:39:16 - lookup material_key.0 - java.lang.RuntimeException: java.lang.ClassCastException: class java.util.Date cannot be cast to class java.sql.Timestamp (java.util.Date is in module java.base of loader 'bootstrap'; java.sql.Timestamp is in module java.sql of loader 'platform')
2024/11/14 09:39:16 - lookup material_key.0 - java.lang.ClassCastException: class java.util.Date cannot be cast to class java.sql.Timestamp (java.util.Date is in module java.base of loader 'bootstrap'; java.sql.Timestamp is in module java.sql of loader 'platform')
2024/11/14 09:39:16 - lookup material_key.0 -
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.pipeline.transforms.dimensionlookup.DimensionCache.lookupRow(DimensionCache.java:172)
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.pipeline.transforms.dimensionlookup.DimensionLookup.lookupValues(DimensionLookup.java:421)
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.pipeline.transforms.dimensionlookup.DimensionLookup.processRow(DimensionLookup.java:212)
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.pipeline.transform.RunThread.run(RunThread.java:54)
2024/11/14 09:39:16 - lookup material_key.0 - at java.base/java.lang.Thread.run(Thread.java:1583)
2024/11/14 09:39:16 - lookup material_key.0 - Caused by: java.lang.RuntimeException: java.lang.ClassCastException: class java.util.Date cannot be cast to class java.sql.Timestamp (java.util.Date is in module java.base of loader 'bootstrap'; java.sql.Timestamp is in module java.sql of loader 'platform')
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.pipeline.transforms.dimensionlookup.DimensionCache.compare(DimensionCache.java:230)
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.pipeline.transforms.dimensionlookup.DimensionCache.compare(DimensionCache.java:36)
2024/11/14 09:39:16 - lookup material_key.0 - at java.base/java.util.Collections.indexedBinarySearch(Collections.java:335)
2024/11/14 09:39:16 - lookup material_key.0 - at java.base/java.util.Collections.binarySearch(Collections.java:323)
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.pipeline.transforms.dimensionlookup.DimensionCache.lookupRow(DimensionCache.java:99)
2024/11/14 09:39:16 - lookup material_key.0 - ... 4 more
2024/11/14 09:39:16 - lookup material_key.0 - Caused by: java.lang.ClassCastException: class java.util.Date cannot be cast to class java.sql.Timestamp (java.util.Date is in module java.base of loader 'bootstrap'; java.sql.Timestamp is in module java.sql of loader 'platform')
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.core.row.value.ValueMetaTimestamp.getTimestamp(ValueMetaTimestamp.java:125)
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.core.row.value.ValueMetaTimestamp.getDate(ValueMetaTimestamp.java:72)
2024/11/14 09:39:16 - lookup material_key.0 - at org.apache.hop.pipeline.transforms.dimensionlookup.DimensionCache.compare(DimensionCache.java:204)
2024/11/14 09:39:16 - lookup material_key.0 - ... 8 more`

### Issue Priority

Priority: 2

### Issue Component

Component: Transforms

Contributor guide

Open the contributing guide

Research direction

Start with pipeline/transforms/dimensionlookup/DimensionCache.java, especially lookupRow and compare, then inspect ValueMetaTimestamp.getTimestamp and getDate. Reproduce a Dimension lookup/update using timestamp date_from and date_to fields with caching and cache preloading enabled. Done means the lookup completes without the reported ClassCastException.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.