Conscrypt fails when building grpc-java on Power

I tried building grpc-java on Power by skipping Android and codegen specific stuff as mentioned here - https://github.com/grpc/grpc-java/blob/master/COMPILING.md, I am facing the following errors:

> Task :grpc-interop-testing:test
[jetty-alpn-agent] Could not find a matching alpn-boot JAR for Java version: 11.0.6
io.grpc.testing.integration.Http2OkHttpTest > classMethod FAILED
    java.lang.RuntimeException: Could not create Conscrypt provider
        at io.grpc.internal.testing.TestUtils.installConscryptIfAvailable(TestUtils.java:180)
        at io.grpc.testing.integration.Http2OkHttpTest.loadConscrypt(Http2OkHttpTest.java:63)
        Caused by:
        java.lang.reflect.InvocationTargetException
            at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
            at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.base/java.lang.reflect.Method.invoke(Method.java:566)
            at io.grpc.internal.ConscryptLoader.newProvider(ConscryptLoader.java:78)
            at io.grpc.internal.testing.TestUtils.installConscryptIfAvailable(TestUtils.java:171)
            ... 1 more
            Caused by:
            java.lang.UnsatisfiedLinkError: no conscrypt_openjdk_jni-linux-ppcle_64 in java.library.path: [/usr/java/packages/lib, /usr/lib64, /lib64, /lib, /usr/lib]
306 tests completed, 1 failed, 20 skipped
> Task :grpc-interop-testing:test FAILED