Altinity / Altinity/clickhouse-operator
Add system.user_processes table to metric scraping for clickhouse versions 23.5 and newer
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 2.6k
- Forks
- 574
- Avg merge
- 8d 6h
- Merged PRs (30d)
- 6
Description
It allow to plot per user memory usage. It also have counters metrics (ProfileEvents), but having all metrics per user in prometheus is probably and overkill.
I would scape user memory (i.e. field memory_usage ) - it's important (especially in the context of max_memory_usage_for_user, which sometimes misbehave).
SELECT *
FROM system.user_processes
Row 1:
──────
user: default
memory_usage: 20280
peak_memory_usage: 20312
ProfileEvents: {'Query':999,'SelectQuery':774,'InsertQuery':11,'QueriesWithSubqueries':1050,'SelectQueriesWithSubqueries':836,'FailedQuery':10,'FailedSelectQuery':2,'FailedInsertQuery':8,'QueryTimeMicroseconds':291074819,'SelectQueryTimeMicroseconds':14543907,'InsertQueryTimeMicroseconds':90045780,'OtherQueryTimeMicroseconds':186485132,'FileOpen':1284655,'ReadBufferFromFileDescriptorRead':918300,'ReadBufferFromFileDescriptorReadBytes':1911963707,'WriteBufferFromFileDescriptorWrite':344074,'WriteBufferFromFileDescriptorWriteBytes':17420233,'FileSync':81,'FileSyncElapsedMicroseconds':665642,'ReadCompressedBytes':1836958979,'CompressedReadBufferBlocks':267007,'CompressedReadBufferBytes':18302182187,'OpenedFileCacheHits':4495,'OpenedFileCacheMisses':940660,'OpenedFileCacheMicroseconds':180234,'IOBufferAllocs':1317781,'IOBufferAllocBytes':363233870154,'ArenaAllocChunks':70594,'ArenaAllocBytes':291631104,'FunctionExecute':119922,'TableFunctionExecute':56,'MarkCacheHits':25235,'MarkCacheMisses':1084,'CreatedReadBufferOrdinary':945155,'DiskReadElapsedMicroseconds':58717702,'DiskWriteElapsedMicroseconds':1530726,'NetworkReceiveElapsedMicroseconds':26602,'NetworkSendElapsedMicroseconds':151059,'NetworkReceiveBytes':123182,'NetworkSendBytes':4906096,'InsertedRows':11,'InsertedBytes':791,'DistributedConnectionTries':74,'DistributedConnectionUsable':72,'DistributedConnectionFailTry':2,'SuspendSendingQueryToShard':75,'CompileFunction':3,'CompileExpressionsMicroseconds':577507,'CompileExpressionsBytes':24576,'SelectedParts':8376,'SelectedRanges':8376,'SelectedMarks':461023,'SelectedRows':2885110301,'SelectedBytes':21744060606,'WaitMarksLoadMicroseconds':1669157,'LoadedMarksCount':70572,'LoadedMarksMemoryBytes':301288,'ContextLock':46795,'ContextLockWaitMicroseconds':5312,'RWLockAcquiredReadLocks':2283,'RWLockReadersWaitMilliseconds':3,'PartsLockHoldMicroseconds':16376,'RealTimeMicroseconds':401724887,'UserTimeMicroseconds':48506195,'SystemTimeMicroseconds':23116288,'SoftPageFaults':71168,'HardPageFaults':1362,'OSIOWaitMicroseconds':148370000,'OSCPUWaitMicroseconds':36175758,'OSCPUVirtualTimeMicroseconds':71583568,'OSReadBytes':890511360,'OSWriteBytes':1409347584,'OSReadChars':1946936584,'OSWriteChars':17517544,'CreatedHTTPConnections':204,'CannotWriteToWriteBufferDiscard':567,'QueryProfilerRuns':467,'S3WriteMicroseconds':5838285,'S3WriteRequestsCount':16,'DiskS3WriteMicroseconds':5838285,'DiskS3WriteRequestsCount':16,'S3DeleteObjects':16,'DiskS3DeleteObjects':16,'EngineFileLikeReadFiles':91,'ReadWriteBufferFromHTTPPreservedSessions':192,'ThreadPoolReaderPageCacheMiss':54656,'ThreadPoolReaderPageCacheMissBytes':1865069139,'ThreadPoolReaderPageCacheMissElapsedMicroseconds':3659429,'AsynchronousReadWaitMicroseconds':26165498,'LogInfo':16,'LogWarning':2,'LogError':20}
Row 2:
──────
user: admin
memory_usage: 145144302563
peak_memory_usage: 145587752989
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the Go metric-scraping entry point and the ClickHouse version checks. Verify how system tables and scraped fields are represented, then add memory_usage from system.user_processes for ClickHouse 23.5 and newer without exporting the ProfileEvents counters. Done means per-user memory metrics are exposed and existing tests or validation pass.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clickhouse, go, kubernetes
- Domain
- observability
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100