aws-samples / aws-samples/dbt-glue
--full-refresh (or) -f is not working for Apache HUDI incremental models
- Dominant language
- Python
- Stars
- 147
- Forks
- 96
- Avg merge
- 7h 4m
- Merged PRs (30d)
- 5
Description
### Describe the bug
A clear and concise description of what the bug is. What command did you run? What happened?
Even if the `-f` flag is added - the models always run in incremental mode and do not recreate the tables! Is there any additional configuration required to enable the full-refresh capabilities for Apache HUDI tables?
### Steps To Reproduce
In as much detail as possible, please provide steps to reproduce the issue. Sample data that triggers the issue, example model code, etc is all very helpful here.
### Expected behavior
A clear and concise description of what you expected to happen.
### Screenshots and log output
If applicable, add screenshots or log output to help explain your problem.
### System information
**The output of `dbt --version`:**
```
Core:
- installed: 1.3.2
- latest: 1.4.5 - Update available!
Your version of dbt-core is out of date!
You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation
Plugins:
- glue: 1.3.12 - Up to date!
- spark: 1.3.1 - Update available!
At least one plugin is out of date or incompatible with dbt-core.
You can find instructions for upgrading here:
https://docs.getdbt.com/docs/installation
```
**The operating system you're using:**
Linux (Ubuntu)
**The output of `python --version`:**
```
Python 3.8.16
```
### Additional context
Add any other context about the problem here.
Contributor guide
Assessment
This issue has not been assessed yet.