[design] Distinguish source data and output data changes
Nobody has claimed this yet.
Assessment
- Difficulty
- 5/5
- Estimated time
- Over a week
- Newbie friendliness
- 25/100
Research direction
No files, tests, or entry points are named. Start by locating the entity data model and the processing path that sets last_change, then compare source-update tracking, last_alert or modifiedAt, and listingDate as possible requirements. Done means an agreed design that clearly distinguishes source changes from processing-output changes.
Written by the indexing model from the issue text.
Description
We have a recurring customer concern around timestamps, especially in scenarios where the customer uses the last_change entity field to re-screen their counterparty database against one or many watchlist entities. Right now, last_change is reflective of changes in the outcome of processing in OS, not the last change at source. I think we need to propose one or more additional fields to provide at-source update timestamps separately. Here's some ideas:
- We could make a table in the database which keeps track of at-source first discovery and most recent change.
- We could add a field to entities to include
last_alert. We could also - alternatively - populate themodifiedAtproperty in the data itself. - We may also want to start computing
listingDatefor source lists where this is not explicitly provided.
- Dominant language
- Python
- Stars
- 810
- Forks
- 187
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 133
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
More from opensanctions/opensanctions
-
ru_mfa_sanctions crawler fails with ValueError: Expected 1 elements, got 0 for xpath './/table' Open
Difficulty 2/5 1-3 hours Newbie friendliness 68/100
opensanctions/opensanctions#5588 ·
-
Difficulty 2/5 1-3 hours Newbie friendliness 76/100
opensanctions/opensanctions#5475 ·
-
daily-issues
Difficulty 2/5 1-3 hours Newbie friendliness 78/100
opensanctions/opensanctions#5238 ·
-
Difficulty 1/5 Under an hour Newbie friendliness 72/100
opensanctions/opensanctions#4742 · 1 comment ·
-
daily-issues
Difficulty 2/5 1-3 hours Newbie friendliness 30/100
opensanctions/opensanctions#5700 ·
All issues in opensanctions/opensanctions
Similar issues
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
bancolombia/sentinel#23 ·
-
test md OpenCI
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
-
integration:quickjs org:external priority:backlog topic:code-interpreter topic:middleware type:feature
Difficulty 2/5 1-3 hours Newbie friendliness 74/100
langchain-ai/deepagents#6450 ·
-
bug client
Difficulty 2/5 1-3 hours Newbie friendliness 88/100
-
Difficulty 2/5 1-3 hours Newbie friendliness 74/100