ClickHouse / ClickHouse/clickhouse-go

increased memory usage in native v2 client

Open
#856 7 comments 4 reactions 0 assignees View on GitHub
discuss performance
Dominant language
Go
Stars
3.3k
Forks
680
Avg merge
2d 3h
Merged PRs (30d)
14

Description

### Issue description
Our system of ClickHouse inserters have been using the native `v2.2.0` client with success for several weeks. After upgrading to `v2.4.3` a few days ago, we noticed that the memory and CPU usage increased markedly. Memory usage was roughly 7-10x more, and CPU usage was 6-9x more on our k8 pods.

### Example code
Our inserters collect in memory ~250k messages, morph them each into ClickHouse row batches and call `client.WriteBatch()` after checking to see if a lock table doesn't have any rows. The columns contain array(string), array(uint8) uint8, and string types. And some of the array(string) average about 500 elements.

### Error log
didn't see any insert error logs

### Configuration
*OS:*
pod images are built using `golang:1.19`

I *think* this is the OS:
```
cat /etc/*-release
...
Debian GNU/Linux 11
...
```

*Interface:* E.g. `native`, `database/sql`
native

*Driver version:*
v.2.4.3

*Go version:* run `go version` in your console
1.19.4

*ClickHouse Server version:*
22.8.5.29

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.