apache / apache/druid

add an integration test with extension diretories as symlinks so that we can validate the version bump for commons-io

Open
#11,393 1 comment 0 reactions 0 assignees View on GitHub
Area - Testing Contributions Welcome
Dominant language
Java
Stars
14.1k
Forks
3.8k
Avg merge
1d 19h
Merged PRs (30d)
209

Description

An extension loading issue was found when using commons-io version 2.9.0. In particular When the an extension directory is configured as a symlink, the symlink is not walked properly, and no jars are found, resulting in the extension not being loaded properly. An example of such a configuration for the druid-hdfs-storage extension is shown below:

```javascript
% pwd
/Users/user1/apache-druid-0.22.0-SNAPSHOT/extensions
%
% ls -al
total 0
drwxr-xr-x 29 user1 staff 928 Jun 29 12:38 .
drwxr-xr-x 12 user1 staff 384 Jun 29 12:35 ..
drwxr-xr-x 28 user1 staff 896 Jun 29 12:35 druid-avro-extensions
drwxr-xr-x 13 user1 staff 416 Jun 29 12:35 druid-aws-rds-extensions
drwxr-xr-x 10 user1 staff 320 Jun 29 12:35 druid-azure-extensions
drwxr-xr-x 3 user1 staff 96 Jun 29 12:35 druid-basic-security
drwxr-xr-x 5 user1 staff 160 Jun 29 12:35 druid-bloom-filter
drwxr-xr-x 4 user1 staff 128 Jun 29 12:35 druid-datasketches
drwxr-xr-x 3 user1 staff 96 Jun 29 12:35 druid-ec2-extensions
drwxr-xr-x 4 user1 staff 128 Jun 29 12:35 druid-google-extensions
lrwxr-xr-x 1 user1 staff 49 Jun 29 12:38 druid-hdfs-storage -> /Users/user1/druid-hdfs-storage
drwxr-xr-x 3 user1 staff 96 Jun 29 12:35 druid-histogram
drwxr-xr-x 10 user1 staff 320 Jun 29 12:35 druid-kafka-extraction-namespace
drwxr-xr-x 7 user1 staff 224 Jun 29 12:35 druid-kafka-indexing-service
drwxr-xr-x 11 user1 staff 352 Jun 29 12:35 druid-kerberos
drwxr-xr-x 6 user1 staff 192 Jun 29 12:35 druid-kinesis-indexing-service
drwxr-xr-x 38 user1 staff 1216 Jun 29 12:35 druid-kubernetes-extensions
drwxr-xr-x 5 user1 staff 160 Jun 29 12:35 druid-lookups-cached-global
drwxr-xr-x 7 user1 staff 224 Jun 29 12:35 druid-lookups-cached-single
drwxr-xr-x 20 user1 staff 640 Jun 29 12:35 druid-orc-extensions
drwxr-xr-x 15 user1 staff 480 Jun 29 12:35 druid-pac4j
drwxr-xr-x 24 user1 staff 768 Jun 29 12:35 druid-parquet-extensions
drwxr-xr-x 3 user1 staff 96 Jun 29 12:35 druid-protobuf-extensions
drwxr-xr-x 73 user1 staff 2336 Jun 29 12:35 druid-ranger-security
drwxr-xr-x 13 user1 staff 416 Jun 29 12:35 druid-s3-extensions
drwxr-xr-x 4 user1 staff 128 Jun 29 12:35 druid-stats
drwxr-xr-x 3 user1 staff 96 Jun 29 12:35 mysql-metadata-storage
drwxr-xr-x 4 user1 staff 128 Jun 29 12:35 postgresql-metadata-storage
drwxr-xr-x 3 user1 staff 96 Jun 29 12:35 simple-client-sslcontext
%
```

An issue was filed with the apache commons-io team https://issues.apache.org/jira/browse/IO-741.

We should add an integration test which tests this scenario so that we can bump the version of this dependency in the future with confidence.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the configured extension-directory symlink scenario described in the issue, then locate the extension-loading integration-test entry point. Add coverage that walks the symlink, finds the extension JARs, and loads the extension successfully; the test should provide confidence when upgrading commons-io.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.