Failed to determine whether the table is a hoodie table
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
spark: 3.2.4
hadoop:3.2.4
hudi:0.13.1
create table hudi_cow_nonpcf_tbl (
uuid int,
name string,
price double
) using hudi
tblproperties (
primaryKey = 'uuid'
);
insert into hudi_cow_nonpcf_tbl select 1, 'a1', 20;
spark-sql (default)> insert into hudi_cow_nonpcf_tbl select 1, 'a1', 20;
2023-09-13 11:09:37,256 WARN adapter.Spark3_2Adapter: Failed to determine whether the table is a hoodie table
com.fasterxml.jackson.core.io.JsonEOFException: Unexpected end-of-input: expected close marker for Object (start marker at [Source: (String)"{"type":"struct","fields":[{"name":"_hoodie_commit_time","type":"string","nullable":true,"metadata":{}},{"name":"_hoodie_commit_seqno","type":"string","nullable":true,"metadata":{}},{"name":"_hoodie_record_key","type":"string","nullable":true,"metadata":{}"; line: 1, column: 183])
at [Source: (String)"{"type":"struct","fields":[{"name":"_hoodie_commit_time","type":"string","nullable":true,"metadata":{}},{"name":"_hoodie_commit_seqno","type":"string","nullable":true,"metadata":{}},{"name":"_hoodie_record_key","type":"string","nullable":true,"metadata":{}"; line: 1, column: 257]
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the issue with Spark 3.2.4, Hadoop 3.2.4, and Hudi 0.13.1 using the supplied table-creation and insert commands. Start at the Spark3_2Adapter warning and its Jackson schema parsing path; done means the insert completes without the truncated JSON warning and hoodie-table detection behaves correctly.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100