google / google/googlesql

Failure on repeated JNI connection

Open
#41 4 comments 1 reaction 0 assignees View on GitHub
Dominant language
C++
Stars
2.6k
Forks
260
PR merge metrics
No merged PRs in 30d

Description

Hi! I'm trying to run ZetaSQL analyzer from a Scala macro, to make some query validation at compile time. It works _wonderfully_ on the first compilation, but when I try to change code (or just clean) and compile again it crashes with this error message:

```
[libprotobuf FATAL external/com_google_protobuf/src/google/protobuf/extension_set.cc:93]
Multiple extension registrations for type "google.protobuf.FieldOptions", field number 68711883.

libc++abi.dylib: terminating with uncaught exception of type google::protobuf::FatalException:
Multiple extension registrations for type "google.protobuf.FieldOptions", field number 68711883.
```

This can be reproduced without any macros, just doing a normal runtime call to the ZetaSQL API with the build tool (sbt) trying to reuse the same JVM. It works with _forking_ JVM for each run, but it's not a solution for the original use case (calling ZetaSQL at compile time).

I tried a few things, including reimplementing `JniChannelProvider` in Scala to be able to localize the problem and potentially tweak its behavior. Unfortunately, I don't know what I could (safely) tweak to make it work. As far as I understood the problem is not in the `JniChannelProvider` itself, but in the JNI library being loaded twice. I thought that maybe I can avoid that, so I tried this:

- extract the `.dylib` file and load the library from that stable location (in contrast to how `cz.adamh.utils.NativeUtils` extracts it to a new temp location every time), that led to
> UnsatisfiedLinkError: Native Library [...]/liblocal_service_jni.dylib already loaded in another classloader
- load it conditionally by checking if the classloader already has it (after the first run), that led me to
> UnsatisfiedLinkError: static.experiment.SocketProvider.getSocketChannel()Ljava/nio/channels/SocketChannel;

So from that I'm deducing that the problem is in the native `getSocketChannel` method. I'm hoping to get some insights in the way it works. I tried to look at `zetasql/local_service/local_service_jni.cc` but didn't understand much. Maybe it can be made idempotent to avoid "multiple extension registrations"? Or maybe I'm doing it all wrong and should somehow reuse the connection?

I can provide more details and some code if that helps. I would appreciate any advice or information.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.