[Python][C++] `ArrowKeyError: Attempted to register factory for scheme 'file'` when using pip-installed GDAL

### Describe the bug, including details regarding any error messages, version, and platform.

As always, let me preface this with a thanks for this project and the help the `arrow` developers are giving to the community (which I have benefited from before).

# Description
I've been tracking a more-than-likely install related bug. Here is a minimal reproducer

```python
# file test.py
import osgeo
from pyarrow import fs
local = fs.LocalFileSystem()
```
and if I run it
```bash
$ python test.py
Traceback (most recent call last):
File "test.py", line 3, in
local = fs.LocalFileSystem()
^^^^^^^^^^^^^^^^^^^^
File "pyarrow/_fs.pyx", line 1112, in pyarrow._fs.LocalFileSystem.__init__
File "pyarrow/error.pxi", line 155, in pyarrow.lib.pyarrow_internal_check_status
File "pyarrow/error.pxi", line 92, in pyarrow.lib.check_status
pyarrow.lib.ArrowKeyError: Attempted to register factory for scheme 'file' but that scheme is already registered.
```
Note that if I do not `import osgeo` there is no error.

# Reproduce

The steps from scratch are the following
```bash
conda create -n test-arrow python=3.12
conda activate test-arrow
pip install pyarrow 'gdal==3.9.2'
```

Probably relevant is the fact that I have `gdal` libraries installed with homebrew (on MacOS)
```bash
$ brew info gdal
==> gdal: stable 3.9.2 (bottled), HEAD
Geospatial Data Abstraction Library
https://www.gdal.org/
Conflicts with:
avce00 (because both install a cpl_conv.h header)
cpl (because both install cpl_error.h)
Installed
/opt/homebrew/Cellar/gdal/3.9.2_1 (497 files, 35.9MB) *
Poured from bottle using the formulae.brew.sh API on 2024-09-27 at 14:14:44
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/g/gdal.rb
```

# Workaround

Everything works if I `conda install gdal` instead of using `pip` (presumably, this also install the libraries and does not use the `homebrew`-installed gdal).

### Component(s)

Python