crate / crate/dbt-cratedb2

[Feature] Cleanup leftover "temporary" tables

Open
#25 3 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
6
Forks
0
PR merge metrics
No merged PRs in 30d

Description

## Problem

The adapter currently implements temporary tables as actual tables.
They are not deleted after being used, that's why they start clogging available resources.

## Feature request

When a special naming convention is followed, for example prefixed by `tmp_`, left over tables can be easily identified and dropped. This feature request is to identify a suitable injection point where "old enough" temporary tables could be dropped.

## Alternatives

Manual/scheduled clean-up outside of dbt.

## References
- https://community.cratedb.com/t/cratedb-incremental-model-materialization-not-working-with-dbt-core/2010

Contributor guide

Open the contributing guide

Research direction

Start by tracing how the dbt adapter implements temporary tables and how incremental model materialization completes. Identify where table age and the `tmp_` naming convention could be evaluated, then determine the cleanup policy and injection point. Done should include an agreed mechanism for safely removing old temporary tables, with coverage for the relevant materialization flow.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.