drag and drop useDrag hook use of useDragModality causes too many rerenders

# 🐛 Bug Report

Draggable components using props from useDraggableItem hook rerender 3 times whenever the keyboard/mouse modality changes. This creates long delays when a separate input is clicked into and then typed into when over ~50 draggable items are in the dom.

## 🤔 Expected Behavior

Each draggable component would ideally not rerender if there is no presentational change, or at most once if it's really necessary.

## 😯 Current Behavior

useDragModality ends up added multiple event listeners to the document which is causing the component to rerender multiple times whenever the user switches from mouse to keyboard, even though there is no visiual change in the component.

## 💁 Possible Solution

Only watching for modality changes within the container managing the drag collection might be the easiest but not ideal solution.

## 🔦 Context

I have a list of ~120 draggable items and a form above them. Using the form is extremely slow due to the list rerendering so many times. The slowness is not very noticeable when just using the draggable list because you are usually going to be using the mouse or keyboard exclusively, however the text input fields above it cause the issue to manifest frequently.

## 💻 Code Sample

https://codesandbox.io/s/dnd-base-cy15o
Open the console and observe the number of times "rendering collection item" is logged each time modality is switched.

## 🌍 Your Environment

| Software | Version(s) |
| ---------------- | ---------- |
| react-spectrum | 3.14.1
| Browser | chrome
| Operating System | osx big sur

## 🧢 Your Company/Team

Adobe/Marketo

## 🕷 Tracking Issue (optional)