Future submit()/result() takes very long time to complete with 8MB Python object

**What happened**: Smallish "Hello world" style submit/result takes a long time and sometimes produces memory warnings.

**What you expected to happen**: Fast end-to-end result (milliseconds)

**Minimal Complete Verifiable Example**:

```python
Python 3.9.12 | packaged by conda-forge | (main, Mar 24 2022, 23:23:20)
Type 'copyright', 'credits' or 'license' for more information
IPython 8.3.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: from distributed import Client, LocalCluster

In [2]: cluster = LocalCluster(n_workers=2, threads_per_worker=1, memory_limit='
...: 16GiB')

In [3]: client = Client(cluster)

In [4]: def foo():
...: return [1.5] * 1_000_000
...:

In [5]: %time res = client.submit(foo).result()
CPU times: user 7.44 s, sys: 567 ms, total: 8.01 s
Wall time: 20.9 s

```

**Anything else we need to know?**:

This issue has been happening in various flavors for 2+ years (I think 2019-vintage Dask does not have this issue)

**Environment**:

- Dask version: 2022.1.0
- Python version: 3.9.12
- Operating System: MacOS, Linux
- Install method (conda, pip, source): conda install coiled-runtime (but have observed the issue for years with pip or conda installs of dask and distributed