Zipper assumes single S3 profile
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1.1k
- Forks
- 564
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 29
Description
Looking at the code for other reasons I saw https://github.com/IQSS/dataverse/blob/46860665574a2cd796095f6fc8c5ce96f16f0fc1/src/main/java/edu/harvard/iq/dataverse/FileDownloadServiceBean.java#L562-L575 which appears to remove store names from the locations and replace them with the store type (e.g. s3 or file in most cases). Looking at https://github.com/IQSS/dataverse/pull/6986/files#diff-7770650e35b5e54de11afe7c79f54715bdd20e120a446414a02db4a881c77438 , it appears that this location is then used to find the files for the Zipper util.
I did not test/verify, but my guess is that this works in most cases as all file stores provide an absolute path and s3 stores include the bucket name. However, by ignoring the specific store's info, with S3, if a store uses a different profile, or has a different endpoint URL, or other params, the file won't be retrievable.
This would affect any installation that uses s3 stores from different places that configures the Zipper (e.g. Harvard if/when a Globus-enabled S3 store at Harvard is configured along with other AWS s3 stores.)
Which version of Dataverse are you using?
5.11
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with FileDownloadServiceBean.java at lines 562-575 and the Zipper-related code shown in pull request 6986. Verify how store locations are transformed and whether a zipper configured with S3 stores using different profiles or endpoint URLs can retrieve each file. Done means files from distinct S3 configurations remain retrievable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, java
- Domain
- backend, cloud
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100