[SUPPORT]flink-sql write hudi use TIMESTAMP, when hive query, it get time+8h question, use TIMESTAMP_LTZ, the hive schema is bigint but timestamp
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
**when I use sql with timestamp:**
**CREATE TABLE dwd_ratio_path_company(
id DECIMAL(20, 0),
company_id BIGINT,
shareholder_id STRING,
shareholder_entity_type SMALLINT,
shareholder_name_id BIGINT,
investment_ratio_total DECIMAL(24, 12),
is_controller SMALLINT,
is_ultimate SMALLINT,
is_big_shareholder SMALLINT,
is_controlling_shareholder SMALLINT,
equity_holding_path STRING,
create_time TIMESTAMP(3),
update_time TIMESTAMP(3),
is_deleted SMALLINT,
op_ts TIMESTAMP(3),
PRIMARY KEY (id) NOT ENFORCED
) WITH (
'connector' = 'hudi',
'path' = 'obs://hadoop-obs/hudi_ods/ratio_path_company015',
'table.type' = 'MERGE_ON_READ',
-- index
'index.type' = 'BUCKET',
'hoodie.bucket.index.num.buckets' = '128',
-- write
'write.tasks' = '4',
'write.task.max.size' = '512',
'write.batch.size' = '8',
'write.log_block.size' = '64',
'write.precombine' = 'true',
'write.precombine.field' = 'op_ts',
-- compaction
'compaction.async.enabled' = 'true',
'compaction.delta_commits' = '3',
-- clean
'clean.retain_commits' = '0',
-- hive
'hive_sync.enabled' = 'true',
'hive_sync.db' = 'hudi_ods',
'hive_sync.table' = 'ratio_path_company015',
'hive_sync.metastore.uris' = 'thrift://10.99.202.153:9083',
'hive_sync.mode' = 'hms',
'hive_sync.support_timestamp' = 'true'
);
insert into dwd_ratio_path_company select * from ods_ratio_path_company;**
it appear:
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the SQL table definition and INSERT statement in the report, focusing on TIMESTAMP(3), TIMESTAMP_LTZ, Hudi, and the hive_sync.support_timestamp setting. Compare the resulting Hive schema and queried values shown in the screenshots; the issue is resolved when the timestamp type and timezone behavior are explained or corrected consistently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- sql
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100