dandi / dandi/dandi-archive

Use S3Batch for object tag removal

Open
#2,580 0 comments 0 reactions 0 assignees View on GitHub
embargo
Dominant language
Python
Stars
26
Forks
21
Avg merge
4d 23h
Merged PRs (30d)
15

Description

During the dandiset unembargo process, we remove the `embargoed` tag from all objects associated with that dandiset. This is often times many objects, and can be difficult to achieve while balancing speed and reliability (see https://github.com/dandi/dandi-archive/pull/2549).

[S3Batch](https://docs.aws.amazon.com/AmazonS3/latest/userguide/batch-ops.html) can instead be used to perform these operations, and should be more efficient, with less room for error.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the unembargo workflow and pull request #2549, then read the AWS S3Batch documentation linked in the issue. The work is complete when removing the embargoed tag for a dandiset uses S3 Batch Operations reliably for large object sets, with the relevant behavior verified by the project’s existing checks.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
backend, cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.