aiondemand / aiondemand/AIOD-rest-api

Implementing OAI-PMH Data Retrieval Support

Aperta
#84 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
enhancement low priority
Lingua principale
Python
Stelle
36
Fork
77
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

## Description:
We consider adding support for the Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) in our application. This step aims to significantly improve data accessibility and collaboration within our system, benefiting both users and the research community.

## Why Implement OAI-PMH:

- **Streamlined Data Sharing:** By incorporating OAI-PMH, we streamline the process of sharing data with other services, repositories, and research platforms. This standardized protocol facilitates smoother data exchange, making it easier for users to access and integrate our app's data into their research projects.

- **Community Engagement**: Supporting OAI-PMH demonstrates our commitment to open data practices, encouraging engagement and collaboration within the research community. Researchers and institutions will find it easier to interact with our app and contribute to collective knowledge.

- **Enhanced Visibility**: With OAI-PMH integration, our app gains increased visibility within the research landscape. Researchers are more likely to discover and utilize our data, strengthening our app's reputation and impact in the academic community.

## Understanding the Protocol
OAI-PMH uses a single base URL that acts as an access point for data retrieval. This base URL is specified in the protocol implementation. To retrieve data, users can send specific requests to this base URL, and the server responds with metadata records in a structured XML format. The protocol allows users to fetch specific sets of records based on various parameters, enabling efficient harvesting and targeted data retrieval.

The protocol requires sharing the data in Dublin Core format.

## TODOs
If we want to implement OAI-PMH, we should:

1. Translate all our classes into Dublin Core format
2. Create a OAI-PMH endpoint and serve data. We might be able to use existing packages such as https://github.com/infrae/pyoai to set up an OAI-PMH server, but it's currently not clear how well existing packages fit with our requirements

The first step, translating into Dublin Core, might take around a week for all classes. The second step could take between a couple of days to weeks, depending on the quality and the fit of existing packages.

## Recommendation
We recommend to postpone OAI-PMH integration until it's clear that it will be used by other services, because it will require significant effort. Although other datahubs such as Zenodo and Cellar provide their data in OAI-PMH, it doesn't seem to be a well-known and well-used standard:

- Latest protocol version is [2.0 of 2002-06-14](http://www.openarchives.org/OAI/openarchivesprotocol.html)
- Amount of stars on github projects such as https://github.com/infrae/pyoai (81) and https://github.com/mloesch/sickle (91), that can be used as an OAI-PMH client (so to read the data) is low
- It's unclear how relevant Dublin Core still is. It's an old standard, and seems to be overshadowed recently by standards such as schema.org

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.