AttributeError: 'Semaphore' object has no attribute '_periodic_callback_name'

**What happened**: Ran the test suite, in the ongoing output this appeared:
```python
distributed/tests/test_semaphore.py::test_access_semaphore_by_name
Exception ignored in:
Traceback (most recent call last):
File "/build/python-distributed/src/distributed-2020.12.0/distributed/semaphore.py", line 538, in __del__
if self._periodic_callback_name in self.client._periodic_callbacks:
AttributeError: 'Semaphore' object has no attribute '_periodic_callback_name'
Exception ignored in:
Traceback (most recent call last):
File "/build/python-distributed/src/distributed-2020.12.0/distributed/semaphore.py", line 538, in __del__
if self._periodic_callback_name in self.client._periodic_callbacks:
AttributeError: 'Semaphore' object has no attribute '_periodic_callback_name'
FAILED [ 81%]
```

**What you expected to happen**: Test should likely succeed, but at the very least not show this issue.

**Minimal Complete Verifiable Example**:

```
pytest distributed
```

**Anything else we need to know?**: I’m packaging for Arch Linux if that matters.

**Environment**:

- Dask version: 2020.12.0
- Python version: 3.9
- Operating System: Arch Linux
- Install method (conda, pip, source): source