learningequality / learningequality/morango
Move transfer session FSICs to their own table
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 15
- Forks
- 23
- PR merge metrics
- No merged PRs in 30d
Description
Background
The transfer session model tracks both client and server FSICs within JSON database fields for a directional sync. As time has gone on, these fields have begun to represent a significant amount of data. The data within these fields are then used to generate complex query filters against other tables within the database when queuing data for a sync.
Summary
As someone concerned with performance and scaling of Morango, I'd like to move these FSIC fields to a new, dedicated table, thereby reducing the size of the transfer session table (currently 200+GB). I'd like to take advantage of some performance improvements that can be made by leveraging this data directly within the database when queuing records for syncing.
Deliverables
- A new table
TransferSessionCountershould be created with fields:transfer_sessionFKinstance_idUUIDpartitionnullable text fieldserver_counternullable integerclient_counternullable integer
- The table should be have a unique constraint on
transfer_sessionandinstance_id - API endpoints should serialize the table's data into two separate
client_fsicandserver_fsicfields to maintain compatibility - Queuing operation queries should join against this new table
Notes
Some progress has been made but was done prior to several recent changes to queuing queries https://github.com/learningequality/morango/commit/9f919c997b79171143e953e9f8d65f5a5c8740da
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the transfer session model, its JSON FSIC fields, the API serialization, and the queuing operation queries. Review commit 9f919c997b79171143e953e9f8d65f5a5c8740da alongside the recent queuing changes. Done means the new TransferSessionCounter table and constraint exist, API fields remain compatible, and queuing queries join against the table.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- django, python
- Domain
- api, backend, database, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100