apache / apache/hudi

Persist the HoodieIndex type in the hoodie.properties file #409

Open
#14,425 2 comments 0 reactions 1 assignee Claimed by @LeonxLJX View on GitHub
area:storage from-jira good first issue help-wanted priority:high status:pr-available type:bug
Dominant language
Java
Stars
6.2k
Forks
2.5k
Avg merge
2d 8h
Merged PRs (30d)
111

Description

https://github.com/uber/hudi/issues/409

## JIRA info

- Link: https://issues.apache.org/jira/browse/HUDI-37
- Type: Bug

---

## Comments

07/Aug/19 11:18;taherk77;[~vinoth] Can you please give some background on this? What is required to be done here?;;;

---

07/Aug/19 11:34;vinoth;We write some one time configs into `hoodie.properties` during dataset creation time. Index type is not one of these atm.

This issue's goal is to first understand if it makes sense for the index to be not changeable after dataset creation. If yes, then make changes to enforce that. ;;;

---

07/Aug/19 11:35;vinoth;You can imagine for e.g, if you used BloomIndex all along to write data and suddenly switched to HBaseIndex in a subsequent write, it will introduce duplicates, since HBase will be empty and thus treat everything as inserts (even though looking up through BloomIndex would have revealed the updates) ;;;

---

07/Aug/19 12:45;taherk77;In log file that we write can we also write the index used in the footer? So that in such scenarios we can just match the 2 index and throw an exception. What do you think?;;;

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.