Automattic / Automattic/sensei
Old quiz questions can be submitted after quiz is reset, but they aren't graded correctly
- Dominant language
- PHP
- Stars
- 593
- Forks
- 218
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 53
Description
#### Steps to Reproduce
1. Create a quiz with variable/random questions.
2. Open the quiz in 2 browser tabs.
3. In the 1st tab, complete the quiz questions provided, then reset the quiz. You will have a **new** set of questions.
4. In the 2nd tab (which has the **original** questions), complete the quiz.
5. In the 2nd tab, you'll be brought to the quiz results page.
#### What I Expected
Either:
* The quiz results in the 2nd tab should be graded with the questions submitted in the 2nd tab.
* Or, there should be an error/warning saying that the version of the quiz in the 2nd tab is outdated, and needs to be retaken.
#### What Happened Instead
The quiz grading does not work correctly: It gives the wrong score for the quiz total, and the incorrect/correct answers are not displayed correctly.
#### WordPress / Sensei version
4.9.5 / 1.9.20
#### Browser / OS version
Any
#### Screenshot / Video
I'm not comfortable posting a video from our site on Github, but I can provide one to an email address.
#### Context / Source
This was reported by users who were getting odd scores when they submitted quizzes.
After digging into the code, I learned that a user's answers & `questions_asked` are stored in comment meta.
What is happening is that the `questions_asked` for a user are changed when the quiz is "reset". However, if the user has a previous version of the quiz open (with different questions), they can still submit _those_ questions, and Sensei has trouble grading because it was expecting the "new" questions.
This is hard to put into words, so let me know if you need more info!
Contributor guide
Research direction
Reproduce the two-tab reset scenario and trace how quiz answers and `questions_asked` are stored in comment meta and consumed during grading. Determine whether submissions should use the original question set or be rejected as outdated; done means the score and correct/incorrect answers are consistent, or an outdated-quiz warning is shown.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- php, wordpress
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100