add an integration test with extension diretories as symlinks so that we can validate the version bump for commons-io
- Dominant language
- Java
- Stars
- 14.1k
- Forks
- 3.8k
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 209
Description
An extension loading issue was found when using commons-io version 2.9.0. In particular When the an extension directory is configured as a symlink, the symlink is not walked properly, and no jars are found, resulting in the extension not being loaded properly. An example of such a configuration for the druid-hdfs-storage extension is shown below:
```javascript
% pwd
/Users/user1/apache-druid-0.22.0-SNAPSHOT/extensions
%
% ls -al
total 0
drwxr-xr-x 29 user1 staff 928 Jun 29 12:38 .
drwxr-xr-x 12 user1 staff 384 Jun 29 12:35 ..
drwxr-xr-x 28 user1 staff 896 Jun 29 12:35 druid-avro-extensions
drwxr-xr-x 13 user1 staff 416 Jun 29 12:35 druid-aws-rds-extensions
drwxr-xr-x 10 user1 staff 320 Jun 29 12:35 druid-azure-extensions
drwxr-xr-x 3 user1 staff 96 Jun 29 12:35 druid-basic-security
drwxr-xr-x 5 user1 staff 160 Jun 29 12:35 druid-bloom-filter
drwxr-xr-x 4 user1 staff 128 Jun 29 12:35 druid-datasketches
drwxr-xr-x 3 user1 staff 96 Jun 29 12:35 druid-ec2-extensions
drwxr-xr-x 4 user1 staff 128 Jun 29 12:35 druid-google-extensions
lrwxr-xr-x 1 user1 staff 49 Jun 29 12:38 druid-hdfs-storage -> /Users/user1/druid-hdfs-storage
drwxr-xr-x 3 user1 staff 96 Jun 29 12:35 druid-histogram
drwxr-xr-x 10 user1 staff 320 Jun 29 12:35 druid-kafka-extraction-namespace
drwxr-xr-x 7 user1 staff 224 Jun 29 12:35 druid-kafka-indexing-service
drwxr-xr-x 11 user1 staff 352 Jun 29 12:35 druid-kerberos
drwxr-xr-x 6 user1 staff 192 Jun 29 12:35 druid-kinesis-indexing-service
drwxr-xr-x 38 user1 staff 1216 Jun 29 12:35 druid-kubernetes-extensions
drwxr-xr-x 5 user1 staff 160 Jun 29 12:35 druid-lookups-cached-global
drwxr-xr-x 7 user1 staff 224 Jun 29 12:35 druid-lookups-cached-single
drwxr-xr-x 20 user1 staff 640 Jun 29 12:35 druid-orc-extensions
drwxr-xr-x 15 user1 staff 480 Jun 29 12:35 druid-pac4j
drwxr-xr-x 24 user1 staff 768 Jun 29 12:35 druid-parquet-extensions
drwxr-xr-x 3 user1 staff 96 Jun 29 12:35 druid-protobuf-extensions
drwxr-xr-x 73 user1 staff 2336 Jun 29 12:35 druid-ranger-security
drwxr-xr-x 13 user1 staff 416 Jun 29 12:35 druid-s3-extensions
drwxr-xr-x 4 user1 staff 128 Jun 29 12:35 druid-stats
drwxr-xr-x 3 user1 staff 96 Jun 29 12:35 mysql-metadata-storage
drwxr-xr-x 4 user1 staff 128 Jun 29 12:35 postgresql-metadata-storage
drwxr-xr-x 3 user1 staff 96 Jun 29 12:35 simple-client-sslcontext
%
```
An issue was filed with the apache commons-io team https://issues.apache.org/jira/browse/IO-741.
We should add an integration test which tests this scenario so that we can bump the version of this dependency in the future with confidence.
Contributor guide
Research direction
Start by reproducing the configured extension-directory symlink scenario described in the issue, then locate the extension-loading integration-test entry point. Add coverage that walks the symlink, finds the extension JARs, and loads the extension successfully; the test should provide confidence when upgrading commons-io.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- testing
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100