Idle memory use increasing over time

In the process of debugging some memory issues I noticed that memory usage of a scheduler+worker with no client connection was steadily increasing over time.

Command: `dask-scheduler --no-bokeh & dask-worker localhost:8786 --nthreads 1 --nprocs 120 --memory-limit 3.2GB --no-bokeh &` (default config.yaml)
Result: after a few idle hours, total memory usage went from about 1GB at startup to 4GB (as reported by the Google Cloud dashboard).

I'm aware that there are a lot of subtleties around measuring memory usage on Linux so I'm not sure if this is a real issue or maybe an artifact of the measurement process, but it seemed like a lot of memory for totally inactive processes. Curious if anyone has any thoughts about what might be happening.