internetarchive / internetarchive/openlibrary
MARC dates need normalization to track changing cataloging practices
- Dominant language
- Python
- Stars
- 6.7k
- Forks
- 2k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 138
Description
### Problem
Some time over the past few years (or decade?) library cataloging practice has apparently switched from using "ca." to using "approximately" to designate uncertain dates. Similarly "fl." has morphed into "active" for `florit` / active dates.
You can see the effects of this with the duplicate author record that was created a couple of months ago because the dates weren't normalized, even though they match exactly.
[OL16119383A](https://openlibrary.org/authors/OL16119383A?v=1) Suetonius approximately 69 - approximately 122
[OL68512A](https://openlibrary.org/authors/OL68512A) Suetonius ca. 69 - ca. 122
You can also see that the Library of Congress has switched to the new standard:
https://id.loc.gov/authorities/names/n50053072.html
Term frequency counts for the 31 Jan 2026 dump:
```
14490 ca.
6506 approximately
161 c.
7410 cent.
1088 century
7222 active
2916 fl.
```
Although this affects relatively few authors, it tends to be authors of the classics and also has a knock-on effect in producing duplicate works.
### Reproducing the bug
1. Go to ...
2. Do ...
* Expected behavior: Date terms are normalized to a common representation which can be used for matching, avoiding false negative matches.
* Actual behavior:
### Context
- Browser (Chrome, Safari, Firefox, etc):
- OS (Windows, Mac, etc):
- Logged in (Y/N):
- Environment (prod, dev, local): prod
### Breakdown
Implementation Details (for maintainers)
#### Requirements Checklist
* [ ]
#### Related files
*
#### Stakeholders
* @hornc
#### Instructions for Contributors
- Please [run these commands](https://github.com/internetarchive/openlibrary/wiki/Git-Cheat-Sheet#working-on-your-branch) to ensure your repository is up to date **before** [creating a new branch](https://github.com/internetarchive/openlibrary/wiki/Git-Cheat-Sheet#making-changes-and-creating-a-pull-request) to work on this issue and **each time after** pushing code to Github, because the pre-commit bot may add commits to your PRs upstream.
Contributor guide
Research direction
Start by tracing how Open Library parses and compares author date strings, using the duplicate Suetonius records and the listed MARC terms as examples. Identify the relevant implementation and tests, then define normalization behavior for equivalent terms such as “ca.” and “approximately” so matching no longer creates false duplicates.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100