influxdata / influxdata/docs-v2

Clustered: modifying querier RAM allocation

Open
#6,137 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
82
Forks
326
Avg merge
1d 1h
Merged PRs (30d)
82

Description

INFLUXDB_IOX_EXEC_MEM_POOL_BYTES is [mentioned, but only as an example](https://docs.influxdata.com/influxdb3/clustered/admin/env-vars/#view-example-of-environment-variables-in-all-components). It really shouldn't be edited in isolation. We probably should have some guidance around the topic of adjusting querier RAM allocation:

-----

Why adjust the RAM allocation? The only common use case: Your queriers have underutilized memory and your workload could benefit from caching more parquet (high query latency as a result of continuous or frequent object store accesses). Only adjust in response to empirical evidence. In this case you may want to decrease INFLUXDB_IOX_EXEC_MEM_POOL_BYTES, and increase INFLUXDB_IOX_RAM_POOL_DATA_BYTES by the same amount.

Current allocation of the queriers' memory allocation (requests):

- 50% to INFLUXDB_IOX_EXEC_MEM_POOL_BYTES (query execution)
- 35% to INFLUXDB_IOX_RAM_POOL_DATA_BYTES (in-memory parquet cache)
- 5% to INFLUXDB_IOX_RAM_POOL_METADATA_BYTES (metadata)
- (10% remaining for everything else)

Why not adjust the allocation: the defaults work well for many workloads and ensure stability. To avoid instability, avoid:

- picking numbers that exceed the amount of memory the querier has
- cutting in to the 10% remainder
- reducing the metadata allocation
- eliminating too much headroom in the execution pool; the defaults are intentionally generous here to avoid query failures or OOMs (especially during exceptional circumstances such as increased load from the clients or decreased capacity during rollouts or node replacement)
- speculative changes based on a short period of observation (make you have a good sense of what worst case load looks like)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the linked environment-variable documentation page at docs.influxdata.com/influxdb3/clustered/admin/env-vars/#view-example-of-environment-variables-in-all-components. Add the querier RAM allocation guidance from the issue, including the allocation breakdown, adjustment criteria, and stability warnings. Done means readers can understand when and how to adjust the two pools without editing the execution pool in isolation.

Written by the indexing model from the issue text.

Assessment

Domain
documentation, infrastructure
Issue type
Documentation
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.