Publishing dataset in an external repository
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1.1k
- Forks
- 564
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 29
Description
We have a speific feature request, which I think would worth it to solve it with a general solution.
The original request: if a user create an Arts and Humanities dataset, s/he should be able to publish it as well on an external reporitory called DARIAH Repository.
As we know the slogan "lots of copies keep your stuff safe" I believe it would be a valid and supportable use case to create copies of the dataset into external reporitories.
Here is a suggestion for the user interface:

The backend and the workflow would like something like this:
- there should be an ExternalRepository interface, which declares some basic methods to manage this publication process, such as
getName(): returns the name of the repositorygetUrl(): returns the URL of the repository's starting pagepublish(DatasetVersion datasetVersion): the main method which publish the dataset in the repositoryisActive(): returns if the repository is turned on in the current Dataverse instance (by default all are turned off, the site admin can activate them via configuration
- a number of implementations of this interface which responsible for an individual external repositories. Each might have different configuration settings, for authentication, API key etc. These implemntations are singletons.
- mapping of subjects and these ExternalRepository implementation. It might be an enumeration or a database table
- the authenticated user will see the above screen, where the list of the external repositories are generated based on the subjects and the status of the external repository objects.
Here are some code snippets, to get more details:
Mapping of subjects and repositories:
public enum Subject {
SOCIAL_SCIENCES("Social Sciences", GesisRepository.getInstance()), // a social science repo
MEDICINE("Medicine, Health and Life Sciences"),
EARTH("Earth and Environmental Sciences"),
AGRICULTUR("Agricultural Sciences"),
OTHER("Other"),
COMPUTER("Computer and Information Science"),
HUMANITIES("Arts and Humanities", DariahRepository.getInstance()), // a Digital Humanities repo
ASTRONOMY("Astronomy and Astrophysics"),
BUSINESS("Business and Management"),
LAW("Law"),
ENGINEERING("Engineering"),
MATHEMATICS("Mathematical Sciences"),
CHEMISTRY("Chemistry"),
PHYSICS("Physics")
;
private String name;
// initialize with general repositories, which could be available for all subjects
private List<ExternalRepository> repositories = List.of(HarvardDataverse.getInstance(),
DataverseNo.getInstance());
Subject(String name) {
this.name = name;
}
Subject(String name, ExternalRepository... repositories) {
this(name);
this.repositories.addAll(Arrays.asList(repositories));
}
public static Subject byName(String name) {
for (Subject subject : values())
if (subject.name.equals(name))
return subject;
return null;
}
public String getName() {
return name;
}
public List<ExternalRepository> getRepositories() {
return repositories;
}
}
get the list of active repositories:
public List<ExternalRepository> getActiveExternalRepositories() {
List<ExternalRepository> repositories = new ArrayList<>();
for (String name : getDatasetSubjects()) {
Subject subject = Subject.byName(name);
if (subject != null && subject.getRepositories() != null)
for (ExternalRepository repository : subject.getRepositories())
if (repository.isActive())
repositories.add(repository);
}
return repositories;
}
@pdurbin @qqmyers @poikilotherm @djbrooke @4tikhonov I am interested in your opinion. I have some initial code to prove the concept for myself, but for a PR it needs lots of work. I would invent this time only if this idea meets community's opinion. Otherwise I will create an independent webservice specific for the DARIAH repository.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No files, tests, or entry points are named. Start by reviewing the existing dataset publication workflow and the proposed ExternalRepository interface, then clarify the supported repository scope, configuration model, subject mapping, and user-interface behavior with maintainers before implementation; done means an agreed design and implementation plan.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend-api-design, frontend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100