SPIKE: Come up with a new design for storing access information, that does not rely on an infinitely growing GuestBookResponse table. (via an archiving mechanism?)
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1.1k
- Forks
- 564
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 29
Description
We need to re-engineer how we store, and count downloads and other access calls.
Close to 1/3 of our prod. db appears to be the guestbookresponse table now.
Unlike the actionlogrecord table on which we perform a yearly bloodletting, there is no solution for archiving guestbookresponse under the current setup, since we rely on counting the entries in the table for download counts on files and datasets, for display and/or metrics; and because all the downloads and guestbooks data are supposed to be available for viewing and downloading.
I am assuming that it should be doable to add an archiving mechanism. That could be used to delete N years-worth of the oldest records; while preserving the raw counts for display and metrics. And would provide some form of access to them once they are archived. For example, it can be communicated to the users that only the last M years of access records are easily accessible, and the older records need to be retrieved from S3, perhaps 1 year at a time.
I have some ideas that I will add. But I am encouraging everybody to participate/brainstorm.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing how GuestBookResponse records are stored and counted for downloads, display, and metrics, then compare that with the existing ActionLogRecord archiving approach. Done means an agreed design for reducing table growth while preserving raw counts and a defined way to retrieve older access records.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100