IQSS / IQSS/dataverse

Publishing dataset in an external repository

Open
#8,001 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Feature: Publishing & Versions Type: Suggestion User Role: Depositor
Dominant language
Java
Stars
1.1k
Forks
564
Avg merge
2d 2h
Merged PRs (30d)
29

Description

We have a speific feature request, which I think would worth it to solve it with a general solution.

The original request: if a user create an Arts and Humanities dataset, s/he should be able to publish it as well on an external reporitory called DARIAH Repository.

As we know the slogan "lots of copies keep your stuff safe" I believe it would be a valid and supportable use case to create copies of the dataset into external reporitories.

Here is a suggestion for the user interface:

external-repository

The backend and the workflow would like something like this:

  • there should be an ExternalRepository interface, which declares some basic methods to manage this publication process, such as
    • getName(): returns the name of the repository
    • getUrl(): returns the URL of the repository's starting page
    • publish(DatasetVersion datasetVersion): the main method which publish the dataset in the repository
    • isActive(): returns if the repository is turned on in the current Dataverse instance (by default all are turned off, the site admin can activate them via configuration
  • a number of implementations of this interface which responsible for an individual external repositories. Each might have different configuration settings, for authentication, API key etc. These implemntations are singletons.
  • mapping of subjects and these ExternalRepository implementation. It might be an enumeration or a database table
  • the authenticated user will see the above screen, where the list of the external repositories are generated based on the subjects and the status of the external repository objects.

Here are some code snippets, to get more details:

Mapping of subjects and repositories:

public enum Subject {
  SOCIAL_SCIENCES("Social Sciences", GesisRepository.getInstance()), // a social science repo
  MEDICINE("Medicine, Health and Life Sciences"),
  EARTH("Earth and Environmental Sciences"),
  AGRICULTUR("Agricultural Sciences"),
  OTHER("Other"),
  COMPUTER("Computer and Information Science"),
  HUMANITIES("Arts and Humanities", DariahRepository.getInstance()), // a Digital Humanities repo
  ASTRONOMY("Astronomy and Astrophysics"),
  BUSINESS("Business and Management"),
  LAW("Law"),
  ENGINEERING("Engineering"),
  MATHEMATICS("Mathematical Sciences"),
  CHEMISTRY("Chemistry"),
  PHYSICS("Physics")
  ;

  private String name;
  // initialize with general repositories, which could be available for all subjects
  private List<ExternalRepository> repositories = List.of(HarvardDataverse.getInstance(),
                                                          DataverseNo.getInstance());

  Subject(String name) {
    this.name = name;
  }

  Subject(String name, ExternalRepository... repositories) {
    this(name);
    this.repositories.addAll(Arrays.asList(repositories));
  }

  public static Subject byName(String name) {
    for (Subject subject : values())
      if (subject.name.equals(name))
        return subject;
    return null;
  }

  public String getName() {
    return name;
  }

  public List<ExternalRepository> getRepositories() {
    return repositories;
  }
}

get the list of active repositories:

public List<ExternalRepository> getActiveExternalRepositories() {
    List<ExternalRepository> repositories = new ArrayList<>();
    for (String name : getDatasetSubjects()) {
        Subject subject = Subject.byName(name);
        if (subject != null && subject.getRepositories() != null)
            for (ExternalRepository repository : subject.getRepositories())
                if (repository.isActive())
                    repositories.add(repository);
    }
    return repositories;
}

@pdurbin @qqmyers @poikilotherm @djbrooke @4tikhonov I am interested in your opinion. I have some initial code to prove the concept for myself, but for a PR it needs lots of work. I would invent this time only if this idea meets community's opinion. Otherwise I will create an independent webservice specific for the DARIAH repository.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. Start by reviewing the existing dataset publication workflow and the proposed ExternalRepository interface, then clarify the supported repository scope, configuration model, subject mapping, and user-interface behavior with maintainers before implementation; done means an agreed design and implementation plan.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, frontend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.