openedx / openedx/openedx-platform
Debug toolbar crashes xblock api requests with DjangoUnicodeDecodeError
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 8.2k
- Forks
- 4.4k
- Avg merge
- 6d 18h
- Merged PRs (30d)
- 42
Description
This is on a fresh tutor devstack install - the same environment as #37101
To reproduce:
- Navigate to the course outline of the demo course in Authoring.
- Click "New unit".
The page fails with "We encountered a technical error when loading this page. This might be a temporary issue, so please try again in a few minutes. If the problem persists, please go to the Support Page for help."
In the network console, view the following:
Request Method: GET
Request URL: http://studio.local.openedx.io:8001/xblock/container/block-v1:OpenedX+DemoX+DemoCourse+type@vertical+block@0ced03ea1d094e20ad6e18e8daa119fd
(Also observed similar error for:
- http://studio.local.openedx.io:8001/api/contentstore/v1/container_handler/block-v1:OpenedX+DemoX+DemoCourse+type@vertical+block@782386797bfc439d9bcaf187ab4a02d4
- http://studio.local.openedx.io:8001/api/contentstore/v1/container/vertical/block-v1:OpenedX+DemoX+DemoCourse+type@vertical+block@782386797bfc439d9bcaf187ab4a02d4/children
)
Traceback (most recent call last):
File "/openedx/venv/lib/python3.11/site-packages/debug_toolbar/store.py", line 22, in default
return super().default(o)
^^^^^^^^^^^^^^^^^^
File "/openedx/venv/lib/python3.11/site-packages/django/core/serializers/json.py", line 106, in default
return super().default(o)
^^^^^^^^^^^^^^^^^^
File "/opt/pyenv/versions/3.11.8/lib/python3.11/json/encoder.py", line 180, in default
raise TypeError(f'Object of type {o.__class__.__name__} '
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
During handling of the above exception (Object of type bytes is not JSON serializable), another exception occurred:
File "/openedx/venv/lib/python3.11/site-packages/django/utils/encoding.py", line 70, in force_str
s = str(s, encoding, errors)
^^^^^^^^^^^^^^^^^^^^^^^^
During handling of the above exception ('utf-8' codec can't decode byte 0xbc in position 2: invalid start byte), another exception occurred:
File "/openedx/venv/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
response = get_response(request)
^^^^^^^^^^^^^^^^^^^^^
File "/openedx/venv/lib/python3.11/site-packages/debug_toolbar/middleware.py", line 145, in __call__
return self._postprocess(request, response, toolbar)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/openedx/venv/lib/python3.11/site-packages/debug_toolbar/middleware.py", line 182, in _postprocess
panel.generate_stats(request, response)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/openedx/venv/lib/python3.11/site-packages/debug_toolbar/panels/cache.py", line 212, in generate_stats
self.record_stats(
^
File "/openedx/venv/lib/python3.11/site-packages/debug_toolbar/panels/__init__.py", line 194, in record_stats
self.toolbar.store.save_panel(
^
File "/openedx/venv/lib/python3.11/site-packages/debug_toolbar/store.py", line 122, in save_panel
cls._request_store[request_id][panel_id] = serialize(data)
^^^^^^^^^^^^^^^
File "/openedx/venv/lib/python3.11/site-packages/debug_toolbar/store.py", line 32, in serialize
return json.dumps(data, cls=DebugToolbarJSONEncoder)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/pyenv/versions/3.11.8/lib/python3.11/json/__init__.py", line 238, in dumps
**kw).encode(obj)
^^^^^^^^^^^
File "/opt/pyenv/versions/3.11.8/lib/python3.11/json/encoder.py", line 200, in encode
chunks = self.iterencode(o, _one_shot=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/pyenv/versions/3.11.8/lib/python3.11/json/encoder.py", line 258, in iterencode
return _iterencode(o, 0)
^^^^^^^^^^^^^^^^^
File "/openedx/venv/lib/python3.11/site-packages/debug_toolbar/store.py", line 25, in default
return force_str(o)
^^^^^^^^^^^^
File "/openedx/venv/lib/python3.11/site-packages/django/utils/encoding.py", line 74, in force_str
raise DjangoUnicodeDecodeError(s, *e.args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Exception Type: DjangoUnicodeDecodeError at /xblock/container/block-v1:OpenedX+DemoX+DemoCourse+type@vertical+block@0ced03ea1d094e20ad6e18e8daa119fd
Exception Value: 'utf-8' codec can't decode byte 0xbc in position 2: invalid start byte. You passed in b'x\x01\xbc\xbd\t\xa0\x1c\xd7U&,\xdb\xda\xfc$k\xf1*\'\xc4\xa9\x18/r\xac\'\xd5\xbe(\xb6cY\xb2-\xd9\x96-KJ\xe2Lb:\xb7\xea\xdez\xddV\xbf\xee\x97^$\xbd,\x8c3@p2\r\x93\x81GH\x02$\x01~\x06\x03\x13\x08K \xc0\xf0\x93aK\x80\x0c\x18\xc8\x9f1\xbb\t\xcb\x98-\x98-1\x84\x90\xff\xbbu\xbb\xba\xef{\xdd\xa5{\xab_i\x12\xc7\x91\xde;Uu\xeb\xbb\xa7\xce=\xe7\xdes\xbe\xf3\xe4\xc6o\xdb\xb0\xe1\x92\r\xfc?o_\xd9;\xb8\xac\xd6\xa0+\x83+\xe2n\xbb\xb5\xbf\x1d?\xc1\x92\x1e\xff\xfb\xd6G\xb2?\x1e\xa3+\xdf\xbar\xdb;V\x0eo\xaf\x0f\xae;\xf9M\xbf\xf1\xcd7}\xe5\x9d7\xfc\xd4J<\xd8\xd8i\xb7{+\x83\xdd\xe7\x17\xdb\xb4\xdfd\xfb\xfb\xbdFs\xff\x19\xb6\xdc\xc5\xa5\xf74\xdb\xc9\x99\x07\xd92.\x1dlN\xda\xfdN\x97\x8d\xff\xf0\xce\x95w\xac\x0cv-u\xd8\xd9F\xbb\xdf\xad\x9de\x9dn\xa3\xddZ\xa9o\x1c>\xe5J\xcb}\xff-\xb7\xdf\x7f\xfd\'\xaf\xc2Sv\xb5;\x8d\x85F\x8b4/ ...[clipped]' (<class 'bytes'>)
The large bytestring is zlib compressed data.
The extracted data appears to be a kind of memory dump of the course outline(?):
D@@A@@@@@}(C_idMbson.objectidObjectId)CLhWRΎ$^bDrootQxmodule.util.keysBlockKeyFcourseFcoursePprevious_versionhD)CLhR14&+GYSbPoriginal_versionhD)CLhR14&+GYMb edited_byKD edited_ondatetimedatetimeC
GAAX"MLbson.tz_utilKFixedOffsethW timedeltaK@K@K@RCUTCR}(T_FixedOffset__offseth!R_FixedOffset__nameh"ubRFblocks}(h
...
A@@JjKA@@NjLA@@NjMA@@NjNA@@Nububh
Gproblem ef8c3814d2ce47c6927a8310bba4b849h2)}(h5h6}(Ldisplay_name'Multiple Choice with Hints and FeedbackmarkdownXQC@@<p><img style="width: 100%;" src="/static/Abacus.png" alt="An abacus with two beads pushed up in the last column, 1 bead in the 2nd to last, and one bead pushed down on the top right. " /></p>
>>In the traditional abacus shown above, what number is represented?<<
( ) 3
...
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the failure through the Authoring course outline and read the traceback paths in debug_toolbar/store.py and debug_toolbar/panels/cache.py. Confirm the failing request data is compressed bytes, then verify that the XBlock and contentstore requests complete without the debug toolbar raising DjangoUnicodeDecodeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- django, python
- Domain
- backend, devtools
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100