apache / apache/arrow

Sys.setenv(ARROW_WITH_ZSTD = "ON") not working for install.packages("arrow")

Open
#32,780 0 comments 1 reaction 0 assignees View on GitHub
Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

when I tried to read a feather file, I got a error:

 "NotImplemented: Support for codec 'zstd' not built"

Then, a suggestion prompts:
```java

Error in (function (cond)  :   error in evaluating the argument 'object' in selecting a method for function 'getRanking': NotImplemented: Support for codec 'zstd' not built
In order to read this file, you will need to reinstall arrow with additional features enabled.
Set one of these environment variables before installing:
 * Sys.setenv(LIBARROW_MINIMAL = "false") (for all optional features, including 'zstd')
 * Sys.setenv(ARROW_WITH_ZSTD = "ON") (for just 'zstd')
See https://arrow.apache.org/docs/r/articles/install.html for details
```
I checked the website and do the Sys.setenv() settings as suggested. Then re-installed arrow using install.packages("arrow").  But the ARROW_WITH_ZSTD is still off:
```java

**** arrow with SOURCE_DIR='tools/cpp' BUILD_DIR='/tmp/RtmpCgqXQr/file5e2349e017f8' DEST_DIR='libarrow/arrow-9.0.0' CMAKE='/usr/bin/cmake' EXTRA_CMAKE_FLAGS=' -DARROW_SIMD_LEVEL=NONE -DARROW_RUNTIME_SIMD_LEVEL=NONE' CC='x86_64-conda-linux-gnu-cc' CXX='x86_64-conda-linux-gnu-c++ -std=gnu++11' LDFLAGS='-Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,/home/USER1/anaconda3/envs/r_env/lib -Wl,-rpath-link,/home/USER1/anaconda3/envs/r_env/lib -L/home/USER1/anaconda3/envs/r_env/lib' ARROW_S3='OFF' ARROW_GCS='OFF' ARROW_MIMALLOC='OFF' ARROW_JEMALLOC='OFF' ARROW_JSON='OFF' ARROW_PARQUET='OFF' ARROW_DATASET='OFF' ARROW_WITH_GOOGLE_CLOUD_CPP='OFF' ARROW_WITH_NLOHMANN_JSON='OFF' ARROW_SUBSTRAIT='OFF' ARROW_WITH_PROTOBUF='OFF' ARROW_WITH_BROTLI='OFF' ARROW_WITH_BZ2='OFF' ARROW_WITH_LZ4='OFF' ARROW_WITH_SNAPPY='OFF' ARROW_WITH_ZLIB='OFF' ARROW_WITH_ZSTD='OFF' ARROW_WITH_RE2='OFF' ARROW_WITH_UTF8PROC='OFF'

*****
*****
*****

--   ARROW_WITH_ZSTD=OFF [default=OFF]
--       Build with zstd compression
```
 

Any suggestions for this? Thanks

**Environment**: Ubuntu 20.04.4 LTS (WSL2); R version 4.1.3
**Reporter**: [zpliu](https://issues.apache.org/jira/browse/ARROW-17522)

**Note**: *This issue was originally created as [ARROW-17522](https://issues.apache.org/jira/browse/ARROW-17522). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the installation with Sys.setenv(ARROW_WITH_ZSTD = "ON") followed by install.packages("arrow"), and inspect the displayed CMake configuration for ARROW_WITH_ZSTD. Trace how the R package installation passes environment variables into the CMake build; done means the configuration reports ARROW_WITH_ZSTD=ON and the installed arrow package can read the zstd-compressed Feather file.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, cpp, r
Domain
build-system
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.