keybase / keybase/client

kbfs steadily increases memory usage until system reboots 2nd attempt

Open
#28,740 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Go
Stars
9.2k
Forks
1.3k
Avg merge
12h 58m
Merged PRs (30d)
56

Description

**Describe the bug**
Over the course of ~3 hours, kbfs steadily increases memory usage until it reaches about ~94% usage at which point my system crashes / reboots. During this time every couple of minutes I'm using rsync to copy files onto kbfs.

raspberry pi 5
* 64-bit ARM
* 8 G of RAM
* 2 G of swap memory

I've compiled from source (no standard package for the above). Branch master commit 2763ebd414e3ff95c1373d41405ed8067f82f701

**To Reproduce**
I am running as one specific user which I've used to authenticate my device with my account.

How I start keybase and kbfs:
```
export GOPATH="$HOME/gopath"
export PATH="$PATH:$GOPATH/bin"
export KEYBASE_RUN_MODE=prod

export XDG_RUNTIME_DIR=/run/user/1001

nohup keybase service >> ~/logs/keybase_server.log 2>&1 &

nohup kbfsfuse /home/dllahr/kbfs >> ~/logs/kbfsfuse.log 2>&1 &
```

Here's a log of the results of running top once per minute over the course of ~29 minutes. You can see the memory usage (3rd column from right) goes from 18.0% to 47.4%. In the 6th column you can see the amount of resident memory usage which is also high and steadily increasing - this is not just virtual memory.
```
1337 dllahr 20 0 3937072 1.4g 35104 S 0.0 18.0 3:54.41 kbfsfuse
1337 dllahr 20 0 4071056 1.5g 35104 S 0.0 19.5 4:11.75 kbfsfuse
1337 dllahr 20 0 4140064 1.6g 35104 S 0.0 20.1 4:31.26 kbfsfuse
1337 dllahr 20 0 4141536 1.6g 35104 S 0.0 20.5 4:45.19 kbfsfuse
1337 dllahr 20 0 4275328 1.7g 35104 S 0.0 21.6 5:03.51 kbfsfuse
1337 dllahr 20 0 4342608 1.8g 35104 S 0.0 22.7 5:21.83 kbfsfuse
1337 dllahr 20 0 4411360 1.9g 35104 S 0.0 23.7 5:36.65 kbfsfuse
1337 dllahr 20 0 4479024 1.9g 35104 S 0.0 24.2 5:55.12 kbfsfuse
1337 dllahr 20 0 4614912 2.0g 35104 S 0.0 25.5 6:14.25 kbfsfuse
1337 dllahr 20 0 4614912 2.0g 35104 S 0.0 25.8 6:29.86 kbfsfuse
1337 dllahr 20 0 4749856 2.2g 35104 S 0.0 27.4 6:49.02 kbfsfuse
1337 dllahr 20 0 4817712 2.2g 35104 S 0.0 28.3 7:07.42 kbfsfuse
1337 dllahr 20 0 4886592 2.3g 35104 S 0.0 29.3 7:24.17 kbfsfuse
1337 dllahr 20 0 4954064 2.4g 35104 S 0.0 30.0 7:41.97 kbfsfuse
1337 dllahr 20 0 5021728 2.4g 35104 S 0.0 30.7 8:00.46 kbfsfuse
1337 dllahr 20 0 5155200 2.6g 35104 S 0.0 32.7 8:17.36 kbfsfuse
1337 dllahr 20 0 5223760 2.6g 35104 S 0.0 33.3 8:35.35 kbfsfuse
1337 dllahr 20 0 5355248 2.8g 35104 S 0.0 35.1 8:59.51 kbfsfuse
1337 dllahr 20 0 5422400 2.8g 35152 S 0.0 35.2 9:19.02 kbfsfuse
1337 dllahr 20 0 5488016 2.9g 35152 S 0.0 36.6 9:43.14 kbfsfuse
1337 dllahr 20 0 5557216 2.9g 35152 S 0.0 37.0 10:01.03 kbfsfuse
1337 dllahr 20 0 5754064 3.1g 35152 S 0.0 39.8 10:26.50 kbfsfuse
1337 dllahr 20 0 5754064 3.1g 35152 S 0.0 39.9 10:45.24 kbfsfuse
1337 dllahr 20 0 5955984 3.3g 35152 S 0.0 42.3 11:11.46 kbfsfuse
1337 dllahr 20 0 5960656 3.3g 35152 S 0.0 42.4 11:32.43 kbfsfuse
1337 dllahr 20 0 6026528 3.4g 35152 S 0.0 43.3 11:58.56 kbfsfuse
1337 dllahr 20 0 6026528 3.4g 35152 S 0.0 43.6 12:19.38 kbfsfuse
1337 dllahr 20 0 6358192 3.7g 35152 S 0.0 47.0 12:46.01 kbfsfuse
1337 dllahr 20 0 6364208 3.7g 35152 S 0.0 47.4 13:05.80 kbfsfuse
```

Can you help me track down what's going wrong here? I think keybase has the potential to be an incredibly powerful tool, but this problem makes it hard for me to deploy in a stable, production system.

**Expected behavior**
see above

**Screenshots**
see above

**Additional numbers**
The number 345567

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the memory growth with kbfsfuse and repeated rsync transfers on the Raspberry Pi 5, using the reported top output and logs to measure resident memory. Trace the relevant kbfsfuse behavior to identify the leak; done means sustained transfers no longer cause steadily increasing memory usage or system instability.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, raspberry-pi
Domain
operating-systems, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.