apache / apache/pinot

Quickstart from console throws ClassNotFoundException for PinotS3

Open
#7,211 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
6.1k
Forks
1.5k
Avg merge
1d 21h
Merged PRs (30d)
189

Description

Pinot (Auth)Quickstart throws a ClassNotFoundException for S3PinotFS - and continues regardless. Should S3PinotFS be disabled in quickstart by default?

> alex@alex incubator-pinot % java -cp pinot-distribution/target/pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-standalone/target/pinot-batch-ingestion-standalone-0.8.0-SNAPSHOT.jar org.apache.pinot.tools.AuthQuickstart
> WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
> ***** Starting Zookeeper, controller, broker and server *****
> 2021/07/26 19:23:59.582 ERROR [PinotFSFactory] [main] Could not instantiate file system for class org.apache.pinot.plugin.filesystem.S3PinotFS with scheme s3
> java.lang.ClassNotFoundException: org.apache.pinot.plugin.filesystem.S3PinotFS
> at java.net.URLClassLoader.findClass(URLClassLoader.java:435) ~[?:?]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:589) ~[?:?]
> at org.apache.pinot.spi.plugin.PluginClassLoader.loadClass(PluginClassLoader.java:77) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:294) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:265) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:246) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:51) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.admin.command.QuickstartRunner.registerPinotFS(QuickstartRunner.java:266) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.admin.command.QuickstartRunner.registerDefaultPinotFS(QuickstartRunner.java:256) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.admin.command.QuickstartRunner.startAll(QuickstartRunner.java:167) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.Quickstart.execute(Quickstart.java:187) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.AuthQuickstart.main(AuthQuickstart.java:70) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> 2021/07/26 19:23:59.593 ERROR [QuickstartRunner] [main] Unable to init PinotFS for scheme: s3, class name: org.apache.pinot.plugin.filesystem.S3PinotFS, configs: {region=us-west-2}
> java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.pinot.plugin.filesystem.S3PinotFS
> at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:56) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.admin.command.QuickstartRunner.registerPinotFS(QuickstartRunner.java:266) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.admin.command.QuickstartRunner.registerDefaultPinotFS(QuickstartRunner.java:256) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.admin.command.QuickstartRunner.startAll(QuickstartRunner.java:167) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.Quickstart.execute(Quickstart.java:187) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.AuthQuickstart.main(AuthQuickstart.java:70) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> Caused by: java.lang.ClassNotFoundException: org.apache.pinot.plugin.filesystem.S3PinotFS
> at java.net.URLClassLoader.findClass(URLClassLoader.java:435) ~[?:?]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:589) ~[?:?]
> at org.apache.pinot.spi.plugin.PluginClassLoader.loadClass(PluginClassLoader.java:77) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:294) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:265) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:246) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:51) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> ... 5 more
> 2021/07/26 19:23:59.598 INFO [StartZookeeperCommand] [main] Executing command: StartZookeeper -zkPort 2123 -dataDir /var/folders/tf/zrrx94ld2n76z80y7w53dmr00000gp/T/1627352639507/baseballStats/rawdata/PinotZkDir
> 2021/07/26 19:24:00.824 INFO [StartZookeeperCommand] [main] Start zookeeper at localhost:2123 in thread main
>

Contributor guide

Open the contributing guide

Research direction

Read QuickstartRunner.registerDefaultPinotFS and registerPinotFS, then inspect PinotFSFactory.register. Reproduce the issue with the AuthQuickstart console command shown in the report. Done means the quickstart no longer reports the S3PinotFS ClassNotFoundException while continuing its startup.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.