react-aria focus management causes conflicts with other libraries

### Provide a general summary of the issue here

Perhaps this is just a case of "Yes, this is the way it is, and you should use our ecosystem as much as possible to prevent these sorts of issues," but I thought I'd point it out anyway. We have used a lot of `radix-ui` before `react-aria` really had its feet under it. As a result, a lot of our components are already from that library. But they're missing some components that `react-aria` provides, in this particular case, the `useColorX` hooks. I tried putting a `useColorArea` inside of a `radix-ui` `Popover`, and now it requires two clicks to close the Popover.

### 🤔 Expected Behavior?

I should have the option, when using `react-aria` components to not have the focus management change from other libraries.

### 😯 Current Behavior

When `react-aria` takes over focus, the `radix-ui` `Popover` component requires two clicks to close.

### 💁 Possible Solution

_No response_

### 🔦 Context

I'm not in a place where I'm prepared to switch our Popover component out for the one that comes with `react-aria`, but it does feel like there should be a way to make the two libraries play nice with each other.

### 🖥️ Steps to Reproduce

Steps are listed in the rendered view of the example:
https://codesandbox.io/p/sandbox/react-aria-colorpicker-mwj453?file=%2Fsrc%2FApp.tsx

### Version

"@react-aria/color": "~3.0.0-beta.29",

### What browsers are you seeing the problem on?

Chrome

### If other, please specify.

_No response_

### What operating system are you using?

Windows 11

### 🧢 Your Company/Team

_No response_

### 🕷 Tracking Issue

_No response_