forrtproject / forrtproject/flora-extractor

osf_registration_guid() returns the path segment before the guid, collapsing 55 works onto one identifier

Open
#201 0 comments 0 reactions 0 assignees View on GitHub
stage-3
Dominant language
Python
Stars
2
Forks
1
Avg merge
1d 6h
Merged PRs (30d)
4

Description

**`osf_registration_guid()` reads the path segment in front of a guid as the guid.**
It took the first five-character token after `osf.io/`, and OSF puts several
non-guid segments there:

| URL | returned | correct |
| --- | --- | --- |
| `osf.io/download/hgwkv/` | `download` | `hgwkv` |
| `osf.io/preprints/psyarxiv/abcde` | `preprints` | `abcde` |
| `osf.io/project/ab12d` | `project` | `ab12d` |

**This is an identifier collision, not a cosmetic miss.** `osf_identifier()` keys the
abstract store on the guid, so every pool work whose URL is download-shaped shares the
single key `osf:osf.io/download`. Measured over the pool: **55 works** collapse onto it.
`osf:osf.io/download` is already in `cache/abstracts.sqlite` as a definitive miss —
those 55 were asked about once, between them, and all now carry that one answer.

Here the shared answer was a MISS, so no wrong text was written. Had the fetch
succeeded, one project's description would have been attributed to all 55, and the
checkpoint would have recorded it as each work's own recovered text.

Stage 3 reads the same helper: `acquire_pdf` derives the guid to call
`list_osf_files()`, so a download-shaped URL asks OSF for the files of an object named
"download".

**Fix**: take the first path segment that is not one of the segments OSF puts in front
of a guid, with the preprint shape (`preprints//`) taking the last, and a
version suffix (`d3x9p_v4`) dropped since the versions of one preprint are one record.

**After the fix**, the collided key has to be reopened or those 55 works keep the shared
miss: `abstract_store.drop_misses(["osf"])`, then re-run the OSF backfill phase.

Found while scanning for #200; the same over-capture inflated that scan's groups until
it was caught.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with osf_registration_guid() and trace how osf_identifier() and acquire_pdf derive identifiers and call list_osf_files(). Verify the URL examples, including preprints and version suffixes, then drop the existing OSF misses with abstract_store.drop_misses(["osf"]) and rerun the OSF backfill phase; done means distinct works retain distinct identifiers.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
67/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.