openedx / openedx/openedx-platform

Debug toolbar crashes xblock api requests with DjangoUnicodeDecodeError

Open
#37,102 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
8.2k
Forks
4.4k
Avg merge
6d 18h
Merged PRs (30d)
42

Description

This is on a fresh tutor devstack install - the same environment as #37101

To reproduce:

  1. Navigate to the course outline of the demo course in Authoring.
  2. Click "New unit".

The page fails with "We encountered a technical error when loading this page. This might be a temporary issue, so please try again in a few minutes. If the problem persists, please go to the Support Page for help."

In the network console, view the following:

Request Method: GET
Request URL: http://studio.local.openedx.io:8001/xblock/container/block-v1:OpenedX+DemoX+DemoCourse+type@vertical+block@0ced03ea1d094e20ad6e18e8daa119fd

(Also observed similar error for:

Traceback (most recent call last):
  File "/openedx/venv/lib/python3.11/site-packages/debug_toolbar/store.py", line 22, in default
    return super().default(o)
           ^^^^^^^^^^^^^^^^^^
  File "/openedx/venv/lib/python3.11/site-packages/django/core/serializers/json.py", line 106, in default
    return super().default(o)
           ^^^^^^^^^^^^^^^^^^
  File "/opt/pyenv/versions/3.11.8/lib/python3.11/json/encoder.py", line 180, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

During handling of the above exception (Object of type bytes is not JSON serializable), another exception occurred:
  File "/openedx/venv/lib/python3.11/site-packages/django/utils/encoding.py", line 70, in force_str
    s = str(s, encoding, errors)
        ^^^^^^^^^^^^^^^^^^^^^^^^

During handling of the above exception ('utf-8' codec can't decode byte 0xbc in position 2: invalid start byte), another exception occurred:
  File "/openedx/venv/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/openedx/venv/lib/python3.11/site-packages/debug_toolbar/middleware.py", line 145, in __call__
    return self._postprocess(request, response, toolbar)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/openedx/venv/lib/python3.11/site-packages/debug_toolbar/middleware.py", line 182, in _postprocess
    panel.generate_stats(request, response)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/openedx/venv/lib/python3.11/site-packages/debug_toolbar/panels/cache.py", line 212, in generate_stats
    self.record_stats(
    ^
  File "/openedx/venv/lib/python3.11/site-packages/debug_toolbar/panels/__init__.py", line 194, in record_stats
    self.toolbar.store.save_panel(
    ^
  File "/openedx/venv/lib/python3.11/site-packages/debug_toolbar/store.py", line 122, in save_panel
    cls._request_store[request_id][panel_id] = serialize(data)
                                               ^^^^^^^^^^^^^^^
  File "/openedx/venv/lib/python3.11/site-packages/debug_toolbar/store.py", line 32, in serialize
    return json.dumps(data, cls=DebugToolbarJSONEncoder)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pyenv/versions/3.11.8/lib/python3.11/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
          ^^^^^^^^^^^
  File "/opt/pyenv/versions/3.11.8/lib/python3.11/json/encoder.py", line 200, in encode
    chunks = self.iterencode(o, _one_shot=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/pyenv/versions/3.11.8/lib/python3.11/json/encoder.py", line 258, in iterencode
    return _iterencode(o, 0)
           ^^^^^^^^^^^^^^^^^
  File "/openedx/venv/lib/python3.11/site-packages/debug_toolbar/store.py", line 25, in default
    return force_str(o)
           ^^^^^^^^^^^^
  File "/openedx/venv/lib/python3.11/site-packages/django/utils/encoding.py", line 74, in force_str
    raise DjangoUnicodeDecodeError(s, *e.args)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Exception Type: DjangoUnicodeDecodeError at /xblock/container/block-v1:OpenedX+DemoX+DemoCourse+type@vertical+block@0ced03ea1d094e20ad6e18e8daa119fd
Exception Value: 'utf-8' codec can't decode byte 0xbc in position 2: invalid start byte. You passed in b'x\x01\xbc\xbd\t\xa0\x1c\xd7U&,\xdb\xda\xfc$k\xf1*\'\xc4\xa9\x18/r\xac\'\xd5\xbe(\xb6cY\xb2-\xd9\x96-KJ\xe2Lb:\xb7\xea\xdez\xddV\xbf\xee\x97^$\xbd,\x8c3@p2\r\x93\x81GH\x02$\x01~\x06\x03\x13\x08K \xc0\xf0\x93aK\x80\x0c\x18\xc8\x9f1\xbb\t\xcb\x98-\x98-1\x84\x90\xff\xbbu\xbb\xba\xef{\xdd\xa5{\xab_i\x12\xc7\x91\xde;Uu\xeb\xbb\xa7\xce=\xe7\xdes\xbe\xf3\xe4\xc6o\xdb\xb0\xe1\x92\r\xfc?o_\xd9;\xb8\xac\xd6\xa0+\x83+\xe2n\xbb\xb5\xbf\x1d?\xc1\x92\x1e\xff\xfb\xd6G\xb2?\x1e\xa3+\xdf\xbar\xdb;V\x0eo\xaf\x0f\xae;\xf9M\xbf\xf1\xcd7}\xe5\x9d7\xfc\xd4J<\xd8\xd8i\xb7{+\x83\xdd\xe7\x17\xdb\xb4\xdfd\xfb\xfb\xbdFs\xff\x19\xb6\xdc\xc5\xa5\xf74\xdb\xc9\x99\x07\xd92.\x1dlN\xda\xfdN\x97\x8d\xff\xf0\xce\x95w\xac\x0cv-u\xd8\xd9F\xbb\xdf\xad\x9de\x9dn\xa3\xddZ\xa9o\x1c>\xe5J\xcb}\xff-\xb7\xdf\x7f\xfd\'\xaf\xc2Sv\xb5;\x8d\x85F\x8b4/  ...[clipped]' (<class 'bytes'>)

The large bytestring is zlib compressed data.
The extracted data appears to be a kind of memory dump of the course outline(?):

D@@A@@@@@}(C_idMbson.objectidObjectId)CLhWRΎ$^bDrootQxmodule.util.keysBlockKeyFcourseFcoursePprevious_versionhD)CLhR14&+GYSbPoriginal_versionhD)CLhR14&+GYMb    edited_byKD     edited_ondatetimedatetimeC
GAAX"MLbson.tz_utilKFixedOffsethW       timedeltaK@K@K@RCUTCR}(T_FixedOffset__offseth!R_FixedOffset__nameh"ubRFblocks}(h

...

A@@JjKA@@NjLA@@NjMA@@NjNA@@Nububh
Gproblem ef8c3814d2ce47c6927a8310bba4b849h2)}(h5h6}(Ldisplay_name'Multiple Choice with Hints and FeedbackmarkdownXQC@@<p><img style="width: 100%;" src="/static/Abacus.png" alt="An abacus with two beads pushed up in the last column, 1 bead in the 2nd to last, and one bead pushed down on the top right. " /></p>
>>In the traditional abacus shown above, what number is represented?<<

( ) 3


...

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the failure through the Authoring course outline and read the traceback paths in debug_toolbar/store.py and debug_toolbar/panels/cache.py. Confirm the failing request data is compressed bytes, then verify that the XBlock and contentstore requests complete without the debug toolbar raising DjangoUnicodeDecodeError.

Written by the indexing model from the issue text.

Assessment

Tech stack
django, python
Domain
backend, devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.