pingcap / pingcap/tidb

Optimize memory consumption of take method in topsql stmtstats Aggregator

Open
#65,237 0 comments 0 reactions 0 assignees View on GitHub
type/enhancement
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Enhancement
## Problem Description

Through pprof performance profiling, we discovered that the `aggregator.aggregate` function in the `pkg/util/topsql/stmtstats` package causes extremely high memory consumption when calling the `StatementStats.Take` method, which may make it a memory hotspot in the system.

## Performance Data

Based on pprof flame graph analysis:

### Memory Allocation Space (alloc_space)
- **Function**: `stmtstats.(*StatementStats).Take`
- **Memory Allocation**: 8.48TB (11% of total allocation)
- **Self Allocation**: 8.48TB (100% self-allocated, not from child function calls)

Image

### Memory Allocation Object Count (alloc_objects)
- **Function**: `stmtstats.(*StatementStats).Take`
- **Object Allocation Count**: 194,524,634,791 (28.8% of total allocation)
- **Self Allocation**: 194,524,634,791 (100% self-allocated)

Image

### CPU Time Consumption
- Overall CPU time of `stmtstats.(*aggregator).aggregate` function: 10.32s (4%)
- CPU overhead related to `stmtstats.(*StatementStats).Take` is also significant

Image
Image

## Call Stack

```
stmtstats.(*aggregator).run
└─ stmtstats.(*aggregator).aggregate
└─ sync.(*Map).Range
└─ stmtstats.(*aggregator).aggregate.func1
└─ stmtstats.(*StatementStats).Take ← Memory hotspot
```

## Impact

1. **Memory Pressure**: The `Take` method accounts for over 10% of total memory allocation, which may lead to increased memory pressure in high-concurrency scenarios
2. **GC Pressure**: Large amounts of object allocation will increase garbage collection frequency and duration, affecting overall performance

## Optimization Suggestions

1. **Object Pooling**: Consider using object pools (sync.Pool) to reuse frequently allocated objects in the `Take` method
2. **Batch Processing Optimization**: Check if batch processing logic in the `aggregate` function can be optimized to reduce memory allocation per `Take` call

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.