aws-samples / aws-samples/amazon-redshift-udfs
Redshift can't open shared object file
- Dominant language
- PLpgSQL
- Stars
- 244
- Forks
- 72
- PR merge metrics
- No merged PRs in 30d
Description
I'm trying to install the [h3 library ](https://github.com/uber/h3-py) in Redshift using Pip and the utility install script `installPipModuleAsRedshiftLibrary.sh`.
I am building `h3` on Ubuntu 16.04. I am able to successfully upload the h3.zip file to my S3 bucket.
However, when I try to run the UDF, I get this error
`OSError: /rdsdbdata/user_lib/0/0/288461.zip/h3/out/libh3.so.1: cannot open shared object file: Not a directory`
Full stack trace from `svl_udf_log`:
```line 2, in get_hexbin\n File "__init__.py", line 1, in
File "h3.py", line 39, in
File "__init__.py", line 443, in LoadLibrary
return self._dlltype(name)
File "__init__.py", line 365, in __init__
self._handle = _dlopen(self._name, mode)```
The UDF I'm trying to use:
```CREATE OR REPLACE function get_hexbin(coord_lat double precision, coord_lon double precision, res INTEGER)
RETURNS VARCHAR IMMUTABLE
AS $$
from h3 import h3
return 'hello world'
$$ LANGUAGE plpythonu;
```
While installing `h3` locally, I see this message about skipping wheel but then it uploads a zip file to S3 anyway:
```
Collecting h3
Saved /home/bhavika/Desktop/.h3/h3-3.1.0-cp36-cp36m-linux_x86_64.whl
Skipping h3, due to already being wheel.
```
Contributor guide
Research direction
Start by reading installPipModuleAsRedshiftLibrary.sh and reviewing how the h3 package is imported by the shown Redshift UDF. Reproduce the call and inspect svl_udf_log, comparing the uploaded h3.zip contents with the shared-library path in the traceback. Done means the UDF can import h3 without the libh3.so.1 "Not a directory" error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, shell
- Domain
- database
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100