[R][Python] Unable to load pyarrow package through reticulate in R on windows
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the bug, including details regarding any error messages, version, and platform.
I'm trying to use arrow to share large data frames between python and R. If I load the R packages then try to import the python module this fails with an ImportError from python:
```
> library(arrow)
The tzdb package is not installed. Timezones will not be available to Arrow compute functions.
Attaching package: ‘arrow’
The following object is masked from ‘package:utils’:
timestamp
> library(reticulate)
> py_discover_config()
...
version: 3.9.18 | packaged by conda-forge | (main, Dec 23 2023, 16:29:04) [MSC v.1929 64 bit (AMD64)]
Architecture: 64bit
...
> pa <- import("pyarrow")
Error in py_module_import(module, convert = convert) :
ImportError: DLL load failed while importing lib: The specified procedure could not be found.
Run `reticulate::py_last_error()` for details.
```
Note that I can load the pyarrow in a vanilla python REPL just fine.
If I ensure to import pyarrow before loading the R package then it works, however this is difficult to guarantee (other packages may have already loaded the R arrow package), and causes failures during package development.
```
> library(reticulate)
> import('pyarrow')
Module(pyarrow)
> library(arrow)
The tzdb package is not installed. Timezones will not be available to Arrow compute functions.
Attaching package: ‘arrow’
The following object is masked from ‘package:utils’:
timestamp
```
Because it seems to be driven by some sort of incompatibility between the R and python arrow libraries, I tried various different versions to see if I could get anything that works. Initially I was using R 4.2.1, but could also reproduce with a fresh install of R 4.3.2. I tried various python versions including fresh installs of the latest (3.11.5), and also the conda version as installed by `reticulate::install_miniconda` (3.9.18 above). I tried using the latest version of the pyarrow package (15.0.0), and also downgrading to 14.0.0 to match the latest available R package.
### Component(s)
Python, R
Contributor guide
Assessment
This issue has not been assessed yet.