External Hudi table is changed to Managed after enabling schema on read
- Dominant language
- Java
- Stars
- 6.2k
- Forks
- 2.5k
- Avg merge
- 2d 8h
- Merged PRs (30d)
- 111
Description
### Bug Description
Here’s a clean, GitHub-friendly version using Markdown code fences and collapsible details for better readability:
```markdown
### Verifying Schema-on-Read Behavior in Apache Hudi
#### 1. Create a Hudi table (initially without schema-on-read)
```sql
CREATE TABLE IF NOT EXISTS trips_quickstart1 (
ts BIGINT,
uuid STRING,
rider STRING,
driver STRING,
fare DOUBLE,
city STRING
) USING HUDI
PARTITIONED BY (city)
LOCATION 's3a:///trips_quickstart1'
TBLPROPERTIES (
primaryKey = 'uuid',
preCombineField = 'ts'
);
```
Initial table metadata:
```sql
DESCRIBE EXTENDED trips_quickstart1;
```
```text
+--------------------+---------+-------+
| col_name|data_type|comment|
+--------------------+---------+-------+
| _hoodie_commit_time| string| null|
|_hoodie_commit_seqno| string| null|
| _hoodie_record_key| string| null|
|_hoodie_partition...| string| null|
| _hoodie_file_name| string| null|
| ts| bigint| null|
| uuid| string| null|
| rider| string| null|
| driver| string| null|
| fare| double| null|
| city| string| null|
+--------------------+---------+-------+
Table type: EXTERNAL
Provider: hudi
Location: s3a:///trips_quickstart1
```
#### 2. Enable schema-on-read
```sql
SET hoodie.schema.on.read.enable=true;
```
```text
+-------------------------------+-----+
| key|value|
+-------------------------------+-----+
|hoodie.schema.on.read.enable | true|
+-------------------------------+-----+
```
#### 3. Re-check table metadata after enabling schema-on-read
```sql
DESCRIBE EXTENDED trips_quickstart1;
```
```text
+--------------------+---------+-------+
| col_name|data_type|comment|
+--------------------+---------+-------+
| _hoodie_commit_time| string| null|
|_hoodie_commit_seqno| string| null|
| _hoodie_record_key| string| null|
|_hoodie_partition...| string| null|
| _hoodie_file_name| string| null|
| ts| bigint| null|
| uuid| string| null|
| rider| string| null|
| driver| string| null|
| fare| double| null|
| city| string| null|
+--------------------+---------+-------+
Table type: MANAGED <-- Now reported as MANAGED when schema-on-read is enabled
Provider: hudi
```
**Key observation**: With `hoodie.schema.on.read.enable=true`, Spark reports the Hudi table as `MANAGED` instead of `EXTERNAL`. This is expected behavior in recent Hudi versions when schema evolution/promotion features are active.
```
Paste the above directly into GitHub — it renders cleanly with proper syntax highlighting and collapsible sections if needed.
### Environment
**Hudi version:** 0.15.0
**Query engine:** Spark
**Relevant configs:** `hoodie.schema.on.read.enable`
### Logs and Stack Trace
_No response_
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the SQL CREATE TABLE and DESCRIBE EXTENDED sequence on Hudi 0.15.0 with Spark, first without and then with hoodie.schema.on.read.enable=true. Trace the table-type classification used by DESCRIBE EXTENDED and determine whether the external table should remain EXTERNAL; done means the behavior is clarified and covered by a regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- spark, sql
- Domain
- data-engineering, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100