Altinity / Altinity/clickhouse-operator

Add system.user_processes table to metric scraping for clickhouse versions 23.5 and newer

Open
#1,498 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

stale
Dominant language
Go
Stars
2.6k
Forks
574
Avg merge
8d 6h
Merged PRs (30d)
6

Description

It allow to plot per user memory usage. It also have counters metrics (ProfileEvents), but having all metrics per user in prometheus is probably and overkill.

I would scape user memory (i.e. field memory_usage ) - it's important (especially in the context of max_memory_usage_for_user, which sometimes misbehave).

SELECT *
FROM system.user_processes

Row 1:
──────
user:              default
memory_usage:      20280
peak_memory_usage: 20312
ProfileEvents:     {'Query':999,'SelectQuery':774,'InsertQuery':11,'QueriesWithSubqueries':1050,'SelectQueriesWithSubqueries':836,'FailedQuery':10,'FailedSelectQuery':2,'FailedInsertQuery':8,'QueryTimeMicroseconds':291074819,'SelectQueryTimeMicroseconds':14543907,'InsertQueryTimeMicroseconds':90045780,'OtherQueryTimeMicroseconds':186485132,'FileOpen':1284655,'ReadBufferFromFileDescriptorRead':918300,'ReadBufferFromFileDescriptorReadBytes':1911963707,'WriteBufferFromFileDescriptorWrite':344074,'WriteBufferFromFileDescriptorWriteBytes':17420233,'FileSync':81,'FileSyncElapsedMicroseconds':665642,'ReadCompressedBytes':1836958979,'CompressedReadBufferBlocks':267007,'CompressedReadBufferBytes':18302182187,'OpenedFileCacheHits':4495,'OpenedFileCacheMisses':940660,'OpenedFileCacheMicroseconds':180234,'IOBufferAllocs':1317781,'IOBufferAllocBytes':363233870154,'ArenaAllocChunks':70594,'ArenaAllocBytes':291631104,'FunctionExecute':119922,'TableFunctionExecute':56,'MarkCacheHits':25235,'MarkCacheMisses':1084,'CreatedReadBufferOrdinary':945155,'DiskReadElapsedMicroseconds':58717702,'DiskWriteElapsedMicroseconds':1530726,'NetworkReceiveElapsedMicroseconds':26602,'NetworkSendElapsedMicroseconds':151059,'NetworkReceiveBytes':123182,'NetworkSendBytes':4906096,'InsertedRows':11,'InsertedBytes':791,'DistributedConnectionTries':74,'DistributedConnectionUsable':72,'DistributedConnectionFailTry':2,'SuspendSendingQueryToShard':75,'CompileFunction':3,'CompileExpressionsMicroseconds':577507,'CompileExpressionsBytes':24576,'SelectedParts':8376,'SelectedRanges':8376,'SelectedMarks':461023,'SelectedRows':2885110301,'SelectedBytes':21744060606,'WaitMarksLoadMicroseconds':1669157,'LoadedMarksCount':70572,'LoadedMarksMemoryBytes':301288,'ContextLock':46795,'ContextLockWaitMicroseconds':5312,'RWLockAcquiredReadLocks':2283,'RWLockReadersWaitMilliseconds':3,'PartsLockHoldMicroseconds':16376,'RealTimeMicroseconds':401724887,'UserTimeMicroseconds':48506195,'SystemTimeMicroseconds':23116288,'SoftPageFaults':71168,'HardPageFaults':1362,'OSIOWaitMicroseconds':148370000,'OSCPUWaitMicroseconds':36175758,'OSCPUVirtualTimeMicroseconds':71583568,'OSReadBytes':890511360,'OSWriteBytes':1409347584,'OSReadChars':1946936584,'OSWriteChars':17517544,'CreatedHTTPConnections':204,'CannotWriteToWriteBufferDiscard':567,'QueryProfilerRuns':467,'S3WriteMicroseconds':5838285,'S3WriteRequestsCount':16,'DiskS3WriteMicroseconds':5838285,'DiskS3WriteRequestsCount':16,'S3DeleteObjects':16,'DiskS3DeleteObjects':16,'EngineFileLikeReadFiles':91,'ReadWriteBufferFromHTTPPreservedSessions':192,'ThreadPoolReaderPageCacheMiss':54656,'ThreadPoolReaderPageCacheMissBytes':1865069139,'ThreadPoolReaderPageCacheMissElapsedMicroseconds':3659429,'AsynchronousReadWaitMicroseconds':26165498,'LogInfo':16,'LogWarning':2,'LogError':20}

Row 2:
──────
user:              admin
memory_usage:      145144302563
peak_memory_usage: 145587752989

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the Go metric-scraping entry point and the ClickHouse version checks. Verify how system tables and scraped fields are represented, then add memory_usage from system.user_processes for ClickHouse 23.5 and newer without exporting the ProfileEvents counters. Done means per-user memory metrics are exposed and existing tests or validation pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
clickhouse, go, kubernetes
Domain
observability
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.