dandi / dandi/backups2datalad

Reorganize classes/re-architecture code

Open
#13 0 comments 0 reactions 0 assignees View on GitHub
code quality under consideration
Dominant language
Python
Stars
0
Forks
1
Avg merge
2d 16h
Merged PRs (30d)
5

Description

*WIP design document for a new code architecture*

* Add a `DandisetBackup` class that takes care of all repository-wide operations for backing up a Dandiset
* Primary attributes:
* `dandiset: RemoteDandiset`
* `ds: AsyncDataset`
* `manager: Manager`
* Merge all `Syncer` methods into a single method of this class
* Move the following `Datasetter` methods to this class:
* `update_dandiset()`
* `ensure_github_remote()`
* `sync_dandiset()`
* part of `update_github_metadata()`?
* `tag_releases()`
* `mkrelease()`
* `backup_zarrs()`
* Make `update_dandiset_metadata()` a method of this class
* Make `async_assets()` a method of this class
* Make `register_s3urls()` a method of this class

* Add a `ZarrBackup` class for subsuming `ZarrSyncer`?

* Wrap [this block](https://github.com/dandi/backups2datalad/blob/53478a80b89a712cb5d3dd59224938b5aa844aa9/src/backups2datalad/asyncer.py#L518-L532) in a function that takes a `DandisetBackup`, `aiterassets()` async iterator, `tracker`, & `error_on_change` and returns a structure containing `Report`, `dm.last_timestamp`, `dm.need_add`, & `dm.zarrs`

* Move the `addurl` methods & attributes of `Downloader` to an `Addurl`(?) class, an instance of which will be passed to `Downloader` on instantiation

* Add a `Superdataset` class that takes care of operations on the superdataset

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.