getsentry / getsentry/self-hosted
high resource usage (cpu)
- Dominant language
- Shell
- Stars
- 9.6k
- Forks
- 2k
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 15
Description
### Self-Hosted Version
26.8.0
### CPU Architecture
x86_64
### Docker Version
29.8.0
```
ii docker-ce 5:29.8.0-1~ubuntu.26.04~resolute
```
### Docker Compose Version
v5.1.1
### Machine Specification
- [x] My system meets the minimum system requirements of Sentry
### Installation Type
Fresh install
### Steps to Reproduce
1. I'm doing a fresh install of sentry on a new vps (8 vCPU, 24 GB RAM, 300GB SSD) (Contabo Cloud VPS 8)
( I was coming from an ancient sentry version 9.1.2 :)
2. after installation all works, except resource usage (CPU) is very high, while event insert rate is virtually zero.
(clean installation, no migration)
dedicated vps for sentry (only sentry dockers run on it)
docker compose up --wait takes multiple minutes to start
### Expected Result
Expected: virtual zero cpu usage, as there is nothing to do..
### Actual Result
all 8 cpus, doing 20-30 % each.
load average: 9 to 10 (expected 1-2)
top users seem to be:
* sentry-self-hosted-clickhouse-1
* sentry-self-hosted-seaweedfs-1
* sentry-self-hosted-kafka-1
* sentry-self-hosted-taskworker-1
docker stats (varies over time)
```
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
8522b3b326cb sentry-self-hosted-nginx-1 -- -- / -- -- -- -- --
986a156ba3eb sentry-self-hosted-relay-1 4.45% 176.8MiB / 23.46GiB 0.74% 110MB / 403kB 43.4MB / 0B 21
244319b7d786 sentry-self-hosted-launchpad-taskworker-1 -- -- / -- -- -- -- --
bcf927325073 sentry-self-hosted-sentry-cleanup-1 -- -- / -- -- -- -- --
bd52bd66a282 sentry-self-hosted-symbolicator-cleanup-1 -- -- / -- -- -- -- --
d58f4023c871 sentry-self-hosted-transactions-consumer-1 0.63% 484.2MiB / 23.46GiB 2.02% 864kB / 769kB 1.51MB / 217kB 15
06b6b9c911e6 sentry-self-hosted-events-consumer-1 0.85% 485.9MiB / 23.46GiB 2.02% 974kB / 877kB 2.26MB / 217kB 20
bcbe5b0bb785 sentry-self-hosted-monitors-clock-tasks-1 1.36% 245.9MiB / 23.46GiB 1.02% 844kB / 757kB 451kB / 217kB 6
dec7d3b2c217 sentry-self-hosted-attachments-consumer-1 0.47% 714.8MiB / 23.46GiB 2.97% 878kB / 777kB 1.2MB / 217kB 19
218e0208ac55 sentry-self-hosted-snuba-subscription-consumer-metrics-1 -- -- / -- -- -- -- --
e83bd6a8ec25 sentry-self-hosted-snuba-profiling-functions-consumer-1 2.35% 174.9MiB / 23.46GiB 0.73% 836kB / 750kB 20.5kB / 0B 26
04fc23e90fe5 sentry-self-hosted-snuba-outcomes-billing-consumer-1 -- -- / -- -- -- -- --
97295331777c sentry-self-hosted-snuba-group-attributes-consumer-1 7.83% 311.8MiB / 23.46GiB 1.30% 826kB / 748kB 20.5kB / 0B 37
e3fa81763c70 sentry-self-hosted-snuba-replays-consumer-1 1.20% 176.4MiB / 23.46GiB 0.73% 837kB / 749kB 41kB / 0B 30
bb317a45886c sentry-self-hosted-generic-metrics-consumer-1 2.88% 493.4MiB / 23.46GiB 2.05% 864kB / 774kB 1.61MB / 217kB 20
b32f00e42884 sentry-self-hosted-snuba-outcomes-accepted-consumer-1 1.10% 183.9MiB / 23.46GiB 0.77% 863kB / 767kB 9.43MB / 0B 26
a78165e739cb sentry-self-hosted-snuba-eap-items-consumer-1 0.91% 178.3MiB / 23.46GiB 0.74% 853kB / 768kB 246kB / 0B 34
cd031676ef81 sentry-self-hosted-ingest-feedback-events-1 0.55% 484.1MiB / 23.46GiB 2.02% 861kB / 766kB 2.94MB / 217kB 15
420c687269da sentry-self-hosted-snuba-metrics-consumer-1 -- -- / -- -- -- -- --
d80a1ed63c52 sentry-self-hosted-snuba-errors-consumer-1 1.42% 178.6MiB / 23.46GiB 0.74% 857kB / 747kB 24.6kB / 0B 34
d5402d1af342 sentry-self-hosted-process-segments-1 9.35% 491.6MiB / 23.46GiB 2.05% 872kB / 776kB 475kB / 217kB 21
a2aa44bee428 sentry-self-hosted-snuba-subscription-consumer-eap-items-1 1.89% 181MiB / 23.46GiB 0.75% 824kB / 751kB 2.43MB / 0B 9
952c41cb42b0 sentry-self-hosted-snuba-replacer-1 -- -- / -- -- -- -- --
aaaf0d41189b sentry-self-hosted-process-spans-1 -- -- / -- -- -- -- --
7918a7a71bc0 sentry-self-hosted-snuba-generic-metrics-counters-consumer-1 4.32% 177.4MiB / 23.46GiB 0.74% 832kB / 756kB 41kB / 0B 34
7b3076cad49e sentry-self-hosted-snuba-subscription-consumer-events-1 -- -- / -- -- -- -- --
8756c80e310b sentry-self-hosted-snuba-outcomes-consumer-1 -- -- / -- -- -- -- --
74589d9dac23 sentry-self-hosted-snuba-profiling-profile-chunks-consumer-1 1.59% 177.5MiB / 23.46GiB 0.74% 836kB / 747kB 1.04MB / 0B 26
caada367ba28 sentry-self-hosted-uptime-results-1 0.40% 247.4MiB / 23.46GiB 1.03% 830kB / 741kB 13.8MB / 217kB 6
79f92f5c5f1d sentry-self-hosted-taskworker-1 -- -- / -- -- -- -- --
c3c29f06bf55 sentry-self-hosted-snuba-subscription-consumer-transactions-1 22.29% 176.9MiB / 23.46GiB 0.74% 822kB / 752kB 311kB / 0B 10
7c1fbd83bb20 sentry-self-hosted-snuba-subscription-consumer-generic-metrics-counters-1 0.96% 190.7MiB / 23.46GiB 0.79% 822kB / 759kB 4.11MB / 0B 9
56180503066e sentry-self-hosted-post-process-forwarder-errors-1 2.34% 487.5MiB / 23.46GiB 2.03% 1.75MB / 1.56MB 1.62MB / 217kB 31
128db7cb7f2c sentry-self-hosted-post-process-forwarder-transactions-1 -- -- / -- -- -- -- --
9c2c6a4988c2 sentry-self-hosted-ingest-occurrences-1 1.21% 245.5MiB / 23.46GiB 1.02% 848kB / 760kB 3.82MB / 217kB 6
f943d0b5837b sentry-self-hosted-snuba-profiling-profiles-consumer-1 1.43% 180.4MiB / 23.46GiB 0.75% 849kB / 760kB 3.93MB / 0B 26
06e1df835050 sentry-self-hosted-web-1 -- -- / -- -- -- -- --
cd158291215a sentry-self-hosted-snuba-issue-occurrence-consumer-1 4.09% 308.2MiB / 23.46GiB 1.28% 840kB / 753kB 819kB / 0B 41
3bc88b882dd1 sentry-self-hosted-taskscheduler-1 9.40% 270.3MiB / 23.46GiB 1.13% 315kB / 1.21MB 12.6MB / 217kB 6
91e268029a9c sentry-self-hosted-monitors-clock-tick-1 0.62% 246MiB / 23.46GiB 1.02% 888kB / 840kB 868kB / 217kB 6
bfe915a9ada3 sentry-self-hosted-post-process-forwarder-issue-platform-1 1.64% 485.3MiB / 23.46GiB 2.02% 1.71MB / 1.56MB 8.47MB / 217kB 16
467e6a4e3fcf sentry-self-hosted-ingest-monitors-1 1.83% 246.7MiB / 23.46GiB 1.03% 896kB / 827kB 1.24MB / 217kB 10
72767ef4dcbb sentry-self-hosted-metrics-consumer-1 1.67% 507.3MiB / 23.46GiB 2.11% 915kB / 833kB 37.1MB / 217kB 20
f85ed6c3a11a sentry-self-hosted-snuba-transactions-consumer-1 2.95% 309MiB / 23.46GiB 1.29% 835kB / 747kB 20.5kB / 0B 37
5e357810041d sentry-self-hosted-taskbroker-1 18.45% 24.16MiB / 23.46GiB 0.10% 9.27MB / 8.27MB 3.86MB / 83.5MB 95
3631a4cf15e9 sentry-self-hosted-uptime-checker-1 15.20% 43.31MiB / 23.46GiB 0.18% 48MB / 99.9MB 0B / 0B 15
5d09393fb9b3 sentry-self-hosted-memcached-1 0.27% 16.15MiB / 23.46GiB 0.07% 2.22MB / 35.9MB 225kB / 0B 10
4a852ec4af5a sentry-self-hosted-symbolicator-1 0.03% 75.88MiB / 23.46GiB 0.32% 18.5kB / 126B 3.2MB / 0B 27
40af272013d3 sentry-self-hosted-snuba-api-1 -- -- / -- -- -- -- --
21f848f4e50d sentry-self-hosted-smtp-1 0.00% 15.03MiB / 23.46GiB 0.06% 60.1kB / 38kB 5.04MB / 77.8kB 1
fb62fa4eb32c sentry-self-hosted-pgbouncer-1 0.16% 4.664MiB / 23.46GiB 0.02% 5.62MB / 5.69MB 3.03MB / 0B 1
679e753ed756 sentry-self-hosted-vroom-1 0.02% 10.59MiB / 23.46GiB 0.04% 16.9kB / 126B 2.34MB / 0B 12
8949c16df96a sentry-self-hosted-clickhouse-1 -- -- / -- -- -- -- --
446c40007828 sentry-self-hosted-redis-1 8.06% 10.75MiB / 23.46GiB 0.04% 103MB / 49.5MB 2.11MB / 5.53MB 6
f441760e9170 sentry-self-hosted-kafka-1 -- -- / -- -- -- -- --
d95ca6e2da08 sentry-self-hosted-postgres-1 0.66% 69.24MiB / 23.46GiB 0.29% 3.07MB / 2.46MB 23.7MB / 18.2MB 13
f7eb8aa48c3c sentry-self-hosted-seaweedfs-1 18.94% 101.1MiB / 23.46GiB 0.42% 1.44kB / 426B 2.46MB / 397kB
```
top
```
top - 09:51:50 up 23:40, 7 users, load average: 10.74, 10.45, 11.92
Tasks: 422 total, 2 running, 417 sleeping, 3 stopped, 0 zombie
%Cpu(s): 20.5 us, 13.4 sy, 0.0 ni, 64.0 id, 0.2 wa, 0.0 hi, 1.9 si, 0.0 st
MiB Mem : 24026.7 total, 5023.8 free, 16930.7 used, 2496.5 buff/cache
MiB Swap: 0.0 total, 0.0 free, 0.0 used. 7096.0 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1126 root 20 0 4997992 88284 40348 S 69.6 0.4 299:50.24 containerd
1217 root 20 0 6846388 177372 73744 S 67.3 0.7 307:07.03 dockerd
1806555 ubuntu 20 0 8811868 629100 19664 S 17.2 2.6 9:22.91 java
1887771 root 20 0 1634576 182152 82064 S 15.2 0.7 0:31.93 weed
1806044 uuidd 20 0 8078464 592028 381068 S 14.9 2.4 6:27.21 clickhouse-serv
1892491 root 20 0 43248 34752 12936 S 8.3 0.1 0:00.25 python3
1836860 999 20 0 1539176 549680 32544 S 7.6 2.2 1:09.28 sentry worker-3
1809111 root 20 0 4736100 35064 15820 S 5.6 0.1 2:50.82 taskbroker
1809051 ubuntu 20 0 95272 55496 12736 S 5.0 0.2 3:08.89 uptime-checker
1844667 65532 20 0 1227624 182264 36120 S 5.0 0.7 1:30.22 relay
1806702 999 20 0 42016 12784 5876 S 3.3 0.1 1:51.09 valkey-server
1805764 sentry 20 0 1356252 44808 23100 S 3.0 0.2 0:54.96 docker-compose
1808027 ubuntu 20 0 1050920 207468 31792 S 2.3 0.8 1:04.71 snuba
1808792 ubuntu 20 0 1077580 211020 34472 S 2.0 0.9 1:06.43 snuba
1836436 999 20 0 1667140 325624 63040 S 2.0 1.3 1:05.75 python3
1808487 ubuntu 20 0 1050116 214596 34776 S 1.7 0.9 1:03.13 snuba
1809553 ubuntu 20 0 1086800 211260 34180 S 1.7 0.9 1:05.93 snuba
1809743 ubuntu 20 0 1052168 213912 35100 S 1.7 0.9 1:01.60 snuba
1810397 ubuntu 20 0 995660 213032 35180 R 1.7 0.9 1:07.76 snuba
1818163 999 20 0 1782608 290040 56576 S 1.7 1.2 1:04.88 sentry
1827721 999 20 0 2524028 295472 56028 S 1.7 1.2 1:08.31 sentry
1836414 999 20 0 2035472 325396 62820 S 1.7 1.3 1:13.47 python3
1836415 999 20 0 1665776 324736 62808 S 1.7 1.3 1:06.44 python3
1808242 ubuntu 20 0 964920 210228 34488 S 1.3 0.9 1:01.88 snuba
1808324 ubuntu 20 0 921896 209068 34740 S 1.3 0.8 0:56.44 snuba
1809286 ubuntu 20 0 1003856 212336 34972 S 1.3 0.9 1:08.17 snuba
1809452 ubuntu 20 0 923948 209112 34956 S 1.3 0.8 0:54.99 snuba
1810463 ubuntu 20 0 842756 209364 29192 S 1.3 0.9 0:43.54 snuba
```
IO
```
tal DISK READ: 0.00 B/s | Total DISK WRITE: 46.73 K/s
Current DISK READ: 0.00 B/s | Current DISK WRITE: 280.37 K/s
TID PRIO USER DISK READ DISK WRITE> COMMAND
398 be/3 root 0.00 B/s 35.94 K/s [jbd2/sda1-8]
2261944 be/4 ubuntu 0.00 B/s 7.19 K/s java -Xmx1G -Xms1G -server -XX:+UseG1GC -XX:MaxGCPauseMillis=20 -XX:InitiatingHeapOccupancyPercent=35 -XX:+Expl~/kafka/*:/usr/bin/../share/java/confluent-telemetry/* kafka.Kafka /etc/kafka/kafka.properties [kafka-1001-raft]
96928 be/4 root 0.00 B/s 3.59 K/s dockerd -H fd:// --containerd=/run/containerd/containerd.sock
```
### Event ID
_No response_
Contributor guide
Research direction
Start by reproducing the fresh installation with `docker compose up --wait`, then inspect `docker stats` and `top`, focusing on the listed ClickHouse, SeaweedFS, Kafka, taskworker, and containerd processes. Compare CPU usage while the instance receives no events and identify the responsible service or startup behavior; done means the cause is isolated and idle resource usage is reduced or explained.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clickhouse, docker, docker-compose, kafka, postgres, redis
- Domain
- devops, infrastructure, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100