Quickstart from console throws ClassNotFoundException for PinotS3
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
Pinot (Auth)Quickstart throws a ClassNotFoundException for S3PinotFS - and continues regardless. Should S3PinotFS be disabled in quickstart by default?
> alex@alex incubator-pinot % java -cp pinot-distribution/target/pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:pinot-plugins/pinot-batch-ingestion/pinot-batch-ingestion-standalone/target/pinot-batch-ingestion-standalone-0.8.0-SNAPSHOT.jar org.apache.pinot.tools.AuthQuickstart
> WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
> ***** Starting Zookeeper, controller, broker and server *****
> 2021/07/26 19:23:59.582 ERROR [PinotFSFactory] [main] Could not instantiate file system for class org.apache.pinot.plugin.filesystem.S3PinotFS with scheme s3
> java.lang.ClassNotFoundException: org.apache.pinot.plugin.filesystem.S3PinotFS
> at java.net.URLClassLoader.findClass(URLClassLoader.java:435) ~[?:?]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:589) ~[?:?]
> at org.apache.pinot.spi.plugin.PluginClassLoader.loadClass(PluginClassLoader.java:77) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:294) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:265) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:246) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:51) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.admin.command.QuickstartRunner.registerPinotFS(QuickstartRunner.java:266) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.admin.command.QuickstartRunner.registerDefaultPinotFS(QuickstartRunner.java:256) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.admin.command.QuickstartRunner.startAll(QuickstartRunner.java:167) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.Quickstart.execute(Quickstart.java:187) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.AuthQuickstart.main(AuthQuickstart.java:70) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> 2021/07/26 19:23:59.593 ERROR [QuickstartRunner] [main] Unable to init PinotFS for scheme: s3, class name: org.apache.pinot.plugin.filesystem.S3PinotFS, configs: {region=us-west-2}
> java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.pinot.plugin.filesystem.S3PinotFS
> at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:56) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.admin.command.QuickstartRunner.registerPinotFS(QuickstartRunner.java:266) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.admin.command.QuickstartRunner.registerDefaultPinotFS(QuickstartRunner.java:256) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.admin.command.QuickstartRunner.startAll(QuickstartRunner.java:167) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.Quickstart.execute(Quickstart.java:187) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.tools.AuthQuickstart.main(AuthQuickstart.java:70) [pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> Caused by: java.lang.ClassNotFoundException: org.apache.pinot.plugin.filesystem.S3PinotFS
> at java.net.URLClassLoader.findClass(URLClassLoader.java:435) ~[?:?]
> at java.lang.ClassLoader.loadClass(ClassLoader.java:589) ~[?:?]
> at org.apache.pinot.spi.plugin.PluginClassLoader.loadClass(PluginClassLoader.java:77) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:294) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:265) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.spi.plugin.PluginManager.createInstance(PluginManager.java:246) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> at org.apache.pinot.spi.filesystem.PinotFSFactory.register(PinotFSFactory.java:51) ~[pinot-distribution-0.8.0-SNAPSHOT-shaded.jar:0.8.0-SNAPSHOT-ad5e784c7fde2415e88d730df35346f2bb9282cc]
> ... 5 more
> 2021/07/26 19:23:59.598 INFO [StartZookeeperCommand] [main] Executing command: StartZookeeper -zkPort 2123 -dataDir /var/folders/tf/zrrx94ld2n76z80y7w53dmr00000gp/T/1627352639507/baseballStats/rawdata/PinotZkDir
> 2021/07/26 19:24:00.824 INFO [StartZookeeperCommand] [main] Start zookeeper at localhost:2123 in thread main
>
Contributor guide
Research direction
Read QuickstartRunner.registerDefaultPinotFS and registerPinotFS, then inspect PinotFSFactory.register. Reproduce the issue with the AuthQuickstart console command shown in the report. Done means the quickstart no longer reports the S3PinotFS ClassNotFoundException while continuing its startup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100