creativecommons / creativecommons/quantifying
Integrating PubMed Central As a DataSource to Improve Quantifying Creative Commons
- Dominant language
- Python
- Stars
- 48
- Forks
- 74
- PR merge metrics
- No merged PRs in 30d
Description
## Problem
Improving Quantifying Creative commons with Medical and life Science records from PubMed.
## Description
This Issue aims to integrate PubMed as a new data source for improving Quantifying Creative Commons project. PubMed provides access to biomedical literature with Creative Commons licensing information, contributing valuable insights into open access scientific publishing trends.
## Additional context
### API Documentation
- **[NCBI E-utilities Overview](https://www.ncbi.nlm.nih.gov/books/NBK25501/)** - Complete guide to Entrez Programming Utilities
- **[ESearch Documentation](https://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.ESearch)** - Search parameter reference and examples
- **[EFetch Documentation](https://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.EFetch)** - Data retrieval specifications and XML schemas
- **[PubMed Search Field Descriptions](https://pubmed.ncbi.nlm.nih.gov/help/#search-field-descriptions-and-tags)** - Complete field tag reference
- **[E-utilities Usage Guidelines](https://www.ncbi.nlm.nih.gov/books/NBK25497/)** - Rate limits, best practices, and compliance requirements
## API Limitations and Constraints
### Rate Limiting
- **Maximum Rate**: 3 requests per second (0.34-second intervals)
- **Enforcement**: Implemented via `time.sleep(0.34)` between requests
- **Retry Strategy**: 3 retries with exponential backoff for failed requests
### Data Retrieval Limits
- **Per Request**: Maximum 9,999 records per `esearch` call
- **Batch Size**: 200 papers per `efetch` request for optimal performance
- **Total Limit**: Configurable via `--fetch-limit` parameter (default: 5,000)
### PubMed Data Source Information
- **[PubMed Homepage](https://pubmed.ncbi.nlm.nih.gov/)** - Main search interface and database statistics
- **[PubMed Data Distribution](https://www.nlm.nih.gov/databases/download/pubmed_medline.html)** - Bulk data access and licensing information
- **[MEDLINE/PubMed Data Element Descriptions](https://www.nlm.nih.gov/bsd/mms/medlineelements.html)** - Complete metadata field specifications
- **[PubMed Central (PMC)](https://www.ncbi.nlm.nih.gov/pmc/)** - Full-text article repository with CC license information
- **[NIH Public Access Policy](https://publicaccess.nih.gov/)** - Mandate requiring open access for NIH-funded research
## Implementation
- [x] I would be interested in implementing this feature.
Contributor guide
Assessment
This issue has not been assessed yet.