openedx / openedx/openedx-platform
Invalid characters in unit title break the course structure
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.2k
- Forks
- 4.4k
- Avg merge
- 6d 18h
- Merged PRs (30d)
- 42
Description
Description
When copy-pasting text from some sources into the section, subsection or unit titles, some invisible characters may be inadvertently inserted. When the backend tries to insert the record, the database will reject the operation resulting in a broken course structure.
How to reproduce
In a python console, create a string with a '\u200b' character at the end. Print the result (the special character will not be visible, but will still be there). Select the printed text and copy to the clipboard. Then paste into a section title. Watch the CMS logs.
>>> t='Test\u200b'
>>> t
'Test\u200b'
>>> print(t)
Test
CMS logs:
[2022-11-16 15:16:20,036: ERROR/ForkPoolWorker-43] cms.djangoapps.contentstore.tasks.update_outline_from_modulestore_task[9288311e-d729-4133-9eca-19378df3aff5]: Could not create course outline for course course-v1:TEST+TEST+2022
Traceback (most recent call last):
File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 581, in get_or_create
return self.get(**kwargs), False
File "/openedx/venv/lib/python3.8/site-packages/django/db/models/query.py", line 435, in get
raise self.model.DoesNotExist(
openedx.core.djangoapps.content.learning_sequences.models.LearningSequence.DoesNotExist: LearningSequence matching query does not exist.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
return self.cursor.execute(sql, params)
File "/openedx/venv/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 73, in execute
return self.cursor.execute(query, args)
File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute
res = self._query(query)
File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query
db.query(q)
File "/openedx/venv/lib/python3.8/site-packages/MySQLdb/connections.py", line 254, in query
_mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (1366, "Incorrect string value: '\\xE2\\x80\\x8B' for column 'title' at row 1")
Expected result
Invalid characters should be removed from the titles before inserting into the backend.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start from the logged cms.djangoapps.contentstore.tasks.update_outline_from_modulestore_task entry point and trace how section, subsection, or unit titles reach the database. Reproduce the issue with a title containing \u200b, then add coverage showing invalid characters are removed and the course structure is created successfully.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100