Non-deterministic creation of perseus exercises

• ricecooker version: 0.6.40
• Python version: 3.5.3
• Operating System: Linux

Description

Running Khan Academy zh-CN and it with nearly identical source trees resulted in all the exercises being marked as different and having to re-import them.

What I Did

• Run KA chef, DEPLOY, PUBLISH
• Import in Kolibri
• Re-run KA chef, DEPLOY, PUBLISH
• Update channel in Kolibri

Expected

Only one or two nodes to have changed (the topics nodes modified).

Actual

Every single exercise had changed and required re-importing.

Possible causes

I suspect two possible cause:

1. this line could be the cause: https://github.com/learningequality/ricecooker/blob/master/ricecooker/classes/questions.py#L275 and since running on Python 3.5 the dict orders are not guaranteed to be consistent between different runs.

2. The other possible cause is the .perseus file generation on Studio could be non-deterministic, see https://github.com/learningequality/studio/blob/develop/contentcuration/contentcuration/utils/publish.py#L342-L359 which uses create predictable zip code code slightly different from the predictable zip used in riececooker.

Real life consequences

Khan Academy channel users will need to redownload many files (small but still many) every time a new channel is published, even though exercises are substantially the same.

Reproduce the repeated Khan Academy chef, DEPLOY, and PUBLISH workflow, then compare the generated exercise artifacts. Inspect ricecooker/classes/questions.py around line 275 and ricecooker/utils/zip.py, alongside Studio's contentcuration/utils/publish.py lines 342-359 and 445-450. Done means equivalent exercises produce deterministic outputs and avoid unnecessary re-imports.