mitodl / mitodl/micromasters

Dashboard tasks to refresh edX cache are not properly locked

Open
#2,139 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
32
Forks
17
Avg merge
2d 1h
Merged PRs (30d)
4

Description

The technique used to implement the lock is not right (the id is way too unique) and it is not safe against crashes of the celery task.

Implementing the lock in the right way also would lead towards a celery task that locks until its subtasks are done: this should be avoided.

We should re-think about how to implement this properly

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the dashboard tasks that refresh the edX cache and reviewing the current lock implementation. Determine how the lock behaves when a Celery task crashes and how subtasks affect lock duration; done means the locking approach is safe and does not hold a parent task lock until its subtasks finish.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, distributed-systems
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.