influxdata / influxdata/influxdb

[v3] Hard-Deleted table not reclaiming disk space in InfluxDB 3 Core

Open
#27,200 12 comments 0 reactions 0 assignees View on GitHub
v3
Dominant language
Rust
Stars
31.7k
Forks
3.7k
Avg merge
13h 37m
Merged PRs (30d)
8

Description

Our goal is to permanently remove table data in order to reclaim disk space on resource-constrained edge devices. If the described behavior is expected, we are looking for the recommended way to achieve this.
Retention policies are not suitable for our schema and use case.

__Steps to reproduce:__
List the minimal actions needed to reproduce the behaviour.

1. Start InfluxDB 3 Core (Serverless) version influxdb-3.8-core using the Docker configuration provided below.
2. Create a database and a table.
3. Insert data into the table until noticeable disk usage is observed.
4. Delete the table via the CLI using the command:
```bash
influxdb3 delete table my_table \
--database my_database \
--hard-delete now \
--host http://localhost:8181 \
--token my_token
```
5. Query the database data and metadata, and observe disk usage on the host.

__Expected behaviour:__
The specified table and its underlying data should be immediately and permanently removed. Disk usage on the host should decrease accordingly, as the data is no longer stored.

__Actual behaviour:__
After executing the delete command:
- The original table (my_table) is no longer queryable.
- Disk usage on the host remains unchanged.
- A new table named my_table_ appears.
- Querying this new table returns all of the previously “deleted” data.

This behavior was observed consistently when using either filesystem-based storage or S3 object storage (MinIO).

__Config:__
Docker Compose Configuration used:
```bash
influxdb-test:
image: influxdb:3.8-core
container_name: influxdb-test
ports:
- "8181:8181"
volumes:
- ./influxdb-data:/var/lib/influxdb3
command:
- influxdb3
- serve
- --without-auth
- --node-id=testnode
- --object-store=file
- --data-dir=/var/lib/influxdb3
```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the issue with the provided Docker configuration and the `influxdb3 delete table` CLI command, then inspect the hard-delete behavior. Done means the table and its underlying data are no longer queryable and disk usage decreases for both filesystem and S3-compatible storage.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.