opensanctions / opensanctions/poliloom
Backfill old supporting quotes to include spaces at element boundaries
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 22
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
After adding separator=" " to BeautifulSoup.get_text(), new quotes extracted by the LLM will have proper spaces at HTML element boundaries (e.g. "In office 27 March 2014" instead of "In office27 March 2014").
Old quotes in the database still have the merged format. The frontend highlighter handles both formats via its boundary space skipping logic, so highlighting still works. But it would be cleaner to normalize the old quotes.
Approach: For each source with archived HTML, re-extract text with get_text(separator=" "), then for each old quote find the corresponding span in the new text and update the property_references.supporting_quotes column.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the code that processes archived HTML and updates the property_references.supporting_quotes column. Re-extract each source with BeautifulSoup get_text(separator=" "), then match each old quote to its span in the new text. Done means archived sources have normalized supporting quotes while existing highlighting continues to work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- database
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100