influxdata / influxdata/docs-v2

Add best practice for custom partition in tag bucket

Open
#5,606 0 comments 0 reactions 0 assignees View on GitHub
InfluxDB v3
Dominant language
JavaScript
Stars
82
Forks
326
Avg merge
1d 1h
Merged PRs (30d)
82

Description

_Describe the issue here._

This request applies to both Cloud Dedicated and Clustered, but I will just to keep the information here for Cloud Dedicated to reduce duplicated words.

I'd like to add the following best practice for tag bucket definition:

# Content

**Best practice for number of tag buckets**: While InfluxDB supports up to 1,000 tag buckets, the higher the number of tag buckets you configure, the worse your query performance may become, depending on how you define custom partition templates for a database or table.

For example: if a table's custom partition templates are defined as follows:

```rust
influxctl table create \
--template-timeformat '%Y-%m-%d' \
--template-tag-bucket customerID,1000 \
DATABASE_NAME \
example-table
```

The values of `customerID` tag are bucketed into 1,000 distinct "buckets" each day, which will generate 1,000 partitions per day for this table. Therefore, one month of data will generate 30k partitions (1,000 buckets/day * 30 days/month). If the retention
policy of this database is one month or longer, such a high number of partitions will significantly show down query performance. We recommend keeping the total number of partitions around 1,000 or below for this table by:
1. Reducing the number of tag buckets.
2. Setting the time part to a longer duration, e.g. `'%Y-%m'`.
3. Shortening the retention policy (i.e. keeping the data for a shorter period).

##### Relevant URLs

Add the information to this page under "Tag bucket part templates": https://docs.influxdata.com/influxdb/cloud-dedicated/admin/custom-partitions/partition-templates/#tag-bucket-part-templates

Or "Partitioning best practices" page: https://docs.influxdata.com/influxdb/cloud-dedicated/admin/custom-partitions/best-practices/

Whatever you think is the best!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the Cloud Dedicated custom partitions pages at the two linked URLs, especially the "Tag bucket part templates" and "Partitioning best practices" sections. Decide which page best fits the proposed guidance, add the example and three recommendations, and verify that the wording applies to both Cloud Dedicated and Clustered documentation where appropriate.

Written by the indexing model from the issue text.

Assessment

Domain
databases, documentation
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.