Keyboard focus is sometimes lost when removing `Tag` from collection

### Provide a general summary of the issue here

Sometimes when removing a `` from a collection, focus returns to the beginning of the document rather than moving to the previous Tag in the collection.

https://github.com/user-attachments/assets/9a5ebbdf-ed91-4e6f-8d92-3ba24d2d1909

### 🤔 Expected Behavior?

Focus should move to the previous `` in the collection.

### 😯 Current Behavior

Sometimes focus moves to the `document`.

### 💁 Possible Solution

_No response_

### 🔦 Context

I am implementing a Select with TagGroup based on this example: https://react-aria.adobe.com/Select#taggroup

I noticed this bug during testing.

### 🖥️ Steps to Reproduce

1. Open the [Select with TagGroup example](https://react-aria.adobe.com/Select#taggroup)
2. Click the "States" label to focus the `[+]` button
3. Press Enter to open the select popover
4. Select two items
5. Press Esc to close the select popover, focus returns to the `[+]` button
6. Press Shift + Tab to select the `X` button of the last tag
7. Press Enter to remove the tag from the selection
8. Press Backspace to remove the remaining from the selection
9. Repeat steps 3 -> 8, the problem usually occurs on the first repetition

### Version

1.14.0

### What browsers are you seeing the problem on?

Microsoft Edge

### If other, please specify.

_No response_

### What operating system are you using?

MacOS

### 🧢 Your Company/Team

_No response_

### 🕷 Tracking Issue

_No response_