apache / apache/druid

Quickstart Tutorial Fails with Extension "druid-s3-extensions" enabled

Open
#12,039 5 comments 0 reactions 0 assignees View on GitHub
Area - Extension
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
2d 58m
Merged PRs (30d)
233

Description

### Affected Version

0.22.0

### Description

Steps to reproduce:
- Download a fresh 0.22.0 installation.
- In `conf/druid/single-server/micro-quickstart/_common/common.runtime.properties` in `druid.extensions.loadList` add `"druid-s3-extensions"`
- Start the quickstart: `./bin/start-micro-quickstart`
- Follow the steps to ingest the wiki-ticker test data from the quickstart: https://druid.apache.org/docs/latest/tutorials/index.html#step-4-load-data
- The job reports `FAILED`. I've attached the logs to this issue ([index_parallel_wiki-ticker_hjmbmoen_2021-12-08T12:22:42.895Z.log](https://github.com/apache/druid/files/7675977/index_parallel_wiki-ticker_hjmbmoen_2021-12-08T12.22.42.895Z.log))

The first exception is this one: `java.lang.reflect.InaccessibleObjectException: Unable to make public long[] com.sun.management.internal.HotSpotThreadImpl.getThreadAllocatedBytes(long[]) accessible: module jdk.management does not "exports com.sun.management.internal" to unnamed module @79dc5318`
but the task continues after that, until: `WARN [main] com.amazonaws.util.EC2MetadataUtils - Unable to retrieve the requested metadata (/latest/dynamic/instance-identity/document). Failed to connect to service endpoint:
com.amazonaws.SdkClientException: Failed to connect to service endpoint:`

I would expect the task to finish successfully.

Contributor guide

Open the contributing guide

Research direction

Start with conf/druid/single-server/micro-quickstart/_common/common.runtime.properties and reproduce the failure using ./bin/start-micro-quickstart and the wiki-ticker tutorial steps. Read the attached index_parallel_wiki-ticker log, focusing on the first exceptions and subsequent task failure. Done means the quickstart ingestion completes successfully with druid-s3-extensions enabled.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, java
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.