internetarchive

internetarchive/iari

Import workflows for the Wikipedia Citations Database

Stars: 13
Forks: 8
Open beginner issues: 0
Indexed issues: 56

Dominant language: Python
License: GPL-3.0
Last GitHub push: Aug 20, 2026
Latest indexed: Sep 17, 2026
Contributing guide: No contributing guide
Code of conduct: No code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Open All

56 open issues indexed

as a developer I want to update pywikibot to 7.3 which has a fix of the logging issue Open

pywikibot

internetarchive/iari#163 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a developer I want to add swagger documentation to all endpoints Open

documentation nice to have

internetarchive/iari#166 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
as a developer I want to use vulture to find orphan and unused code in the project to keep down complexity and help find code to deprecate Open

deprecation nice to have

internetarchive/iari#259 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a developer I want an open data mapping of template names and aliases for CS1 templates in all Wikipedias Open

data extraction mediawiki-dataset multiwiki-support open data

internetarchive/iari#495 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a developer I want to add pre-commit hooks to the configuration Open

configuration continuous integration linting nice to have

internetarchive/iari#510 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a developer I want to use conventional commits and verify it using a hook to improve consistency Open

configuration conventional commits enhancement

internetarchive/iari#511 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a patron I want a new retracted paper API that accepts DOI and sources open-retractions, WD and OpenAlex Open

external api retracted paper

internetarchive/iari#627 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a frontend developer I want an endpoint that gives me details about a publisher so I can show an overview to the patron Open

publisher endpoint

internetarchive/iari#685 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a data consumer I want a list of persons on templates so I can easily show them to the patron in a heatmap, word cloud or list Open

parsing persons

internetarchive/iari#694 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a developer I want to fix the docker configuration so we can productionize IARI Open

docker

internetarchive/iari#697 · 0 comments · 0 reactions · 2 assignees · Indexed Sep 17, 2026
as a developer I want to rewrite using fastapi to get free swagger documentation Open

fastapi major rewrite

internetarchive/iari#710 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a patron I want IARI to know which sections to search for any given language version so I don't have to think Open

all endpoint article endpoint mediawiki-dataset

internetarchive/iari#736 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a developer I would like to deprecate the all/ endpoint now that WARE has passed mvp status to reduce complexity Open

all endpoint deprecation

internetarchive/iari#737 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a patron I want the regex used to find general references in the article endpoint in the json object so I can reproduce the results Open

article endpoint enhancement

internetarchive/iari#742 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a patron I want to have an analysis of references per level 2-section so I can drill down further in the articles and see differences e.g. the history section across Wikipedia language editions Open

internetarchive/iari#744 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a developer i want to rename and rewrite the section regex parameter to avoid confusion Open

article endpoint enhancement IARE breaking change rewrite

internetarchive/iari#745 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
as a developer I want to create a test with 2 identical general references defined in the same section Open

parsing testing

internetarchive/iari#747 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a developer I want to rename "named" -> "empty" to avoid confusion Open

internetarchive/iari#748 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a data consumer I want a list of all the level 2 sections so I can present them to the user Open

article endpoint parsing

internetarchive/iari#749 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
as a data consumer I want a list of all the sections that were present and analyzed for general references so I can show then to the patron Open

article endpoint enhancement

internetarchive/iari#751 · 3 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
as a patron I want a clear error when no url is provided to the pdf/ and xhtml/ endpoints Open

bug usability

internetarchive/iari#761 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a developer I want to read the sections from the mediawiki-dataset and use them to extract general references from the right sections Open

article endpoint mediawiki-dataset

internetarchive/iari#782 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a developer I want a new parse-reference POST endpoint that uses books-ai to convert plain text references with no template to structured data using a LLM Open

books-ai endpoint parse-reference endpoint plain text references

internetarchive/iari#785 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
As a developer I want to switch to testdeadlink from iabot in the check-url endpoint Open

check-url endpoint iabot

internetarchive/iari#786 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026
as a developer I want to run the tests using all python version we support Open

continuous integration testing

internetarchive/iari#790 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a developer I want to deprecate python version 3.8 and 3.9 Open

deprecation python

internetarchive/iari#791 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
As a DevOp I want to download the bash history from AWS weekly to make sure we don't lose it if the server dies Open

AWS instance

internetarchive/iari#798 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
in /reference endpoint, returned json always returns "isbn" property Open

reference endpoint

internetarchive/iari#800 · 1 comment · 1 reaction · 1 assignee · Indexed Sep 17, 2026
Include current revision ID in response Open

internetarchive/iari#809 · 6 comments · 1 reaction · 0 assignees · Indexed Sep 17, 2026
as a developer I want access to a dataset extracted from the citoid configuration in each Wikipedia where citoid is installed Open

isaac idea mediawiki-dataset

internetarchive/iari#815 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
As a developer I want to implement catching errors from the pdf parser and test using corrupted PDFs to make the API more stable Open

bug pdf endpoint

internetarchive/iari#822 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a developer I want to update endpoint output documentation in the readme to include detected_language, url_objects and wikitext Open

documentation

internetarchive/iari#833 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a developer I want more tests of the fld extraction to understand better when it fails Open

first level domain extraction

internetarchive/iari#836 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a developer I want to parse "safe links" like these to avoid confusing the patron Open

safe links parsing

internetarchive/iari#839 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a tool developer I want a new endpoint that I can send a chunk of wikitext for a citation and get a complete analysis back Open

enhancement

internetarchive/iari#842 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
As a patron I want link extraction to support ftp also Open

enhancement pdf endpoint

internetarchive/iari#843 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a patron I want the pdf endpoint to extract all urls from Global Connectivity Report so I can check their status Open

edge case link extraction pdf endpoint

internetarchive/iari#844 · 5 comments · 1 reaction · 1 assignee · Indexed Sep 17, 2026
as a developer I want to make sure we can fetch pdfs from the wayback machine also Open

bug pdf endpoint

internetarchive/iari#846 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a patron I want the pdf endpoint to support fetching pdfs from commons Open

415 bug pdf endpoint

internetarchive/iari#847 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a data consumer I want the pdf endpoint to resolve weird code points found in pdfs Open

internetarchive/iari#849 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a customer I want the pdf endpoint to also try to find links by parsing the xhtml generated by pymupdf Open

enhancement link extraction pdf endpoint

internetarchive/iari#852 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
As a consumer i want the DeSantis pdf file to accurately show all the links in the document Open

parsing pdf endpoint

internetarchive/iari#854 · 2 comments · 1 reaction · 1 assignee · Indexed Sep 17, 2026
as a patron I want the links in the Responsible AI to be found and analyzed correctly by IARI so I can trust it Open

edge case link extraction pdf endpoint

internetarchive/iari#860 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a customer I want to batch check urls using a POST request so I don't have to make a ton of http requests per article Open

batch support check-urls endpoint enhancement wrapper

internetarchive/iari#861 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a customer I want a development instance of IARI so we can experiment there without breaking IARE Open

development instance enhancement

internetarchive/iari#862 · 6 comments · 0 reactions · 1 assignee · Indexed Sep 17, 2026
As a developer i want all tests to succeed in "the wild" and not just select dev environments Open

internetarchive/iari#870 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a devop I want to set up monitoring of iari so I can overview the backend Open

monitoring

internetarchive/iari#872 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
as a devop I want to publish the docker image to dockerhub so others can test it out Open

docker

internetarchive/iari#873 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 17, 2026
As a data consumer i want the /check-url endpoint to accurately cache results Open

bug url checking

internetarchive/iari#875 · 10 comments · 1 reaction · 1 assignee · Indexed Sep 17, 2026
As a data consumer i want check-url to return a status code of 200 for a URL that resolves to an .xlsx file Open

bug check-url endpoint

internetarchive/iari#879 · 1 comment · 0 reactions · 1 assignee · Indexed Sep 17, 2026

A short digest of beginner-friendly GitHub issues.