opensafely-core / opensafely-core/opencodelists

`definition.csv` download URL unhandled exception for unsupported coding systems

Open
#2,443 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
60
Forks
16
Avg merge
4d 12h
Merged PRs (30d)
17

Description

Why are we doing this?

The URL to download definition.csv is still available for codelists with coding systems where we don't support that, leading to a 500 error if the URL is accessed. The impact on real human users is minimal because we don't display a link to that URL in the codelist version page for such codelists.

This Sentry event shows a bot hitting this error. That's because it's a DMD codelist that doesn't support definition download.

https://www.opencodelists.org/codelist/opensafely/asthma-inhaler-salbutamol-medication/2020-04-15/
https://www.opencodelists.org/codelist/opensafely/asthma-inhaler-salbutamol-medication/2020-04-15/definition.csv

This Honeycomb query shows 156 such errors in the past 60 days, all of them estimated to be bots based on the user-agent.

How will we know when it's done?

The view handles such errors gracefully instead of throwing a 500 error.

Example stack trace from local development environment:

2025-04-17T10:06:07.841325Z [error    ] request_failed                 [django_structlog.middlewares.request] code=500 ip=127.0.0.1 request='GET /codelist/opensafely/asthma-inhaler-salbutamol-medication/2020-04-15/definition.csv' request_id=e622272e-9ea5-4578-83b9-e299c5b3f047 user_id=mike
Traceback (most recent call last):
  File "/home/mike/opencodelists/.venv/lib/python3.12/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mike/opencodelists/.venv/lib/python3.12/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mike/opencodelists/codelists/views/decorators.py", line 67, in wrapped_view
    rsp = view_fn(request, version, **view_kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mike/opencodelists/codelists/views/version_download_definition.py", line 13, in version_download_definition
    response.write(clv.definition_csv_data_for_download())
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mike/opencodelists/codelists/models.py", line 888, in definition_csv_data_for_download
    return rows_to_csv_data(present_definition_for_download(self))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mike/opencodelists/codelists/presenters.py", line 31, in present_definition_for_download
    codeset = clv.codeset
              ^^^^^^^^^^^
  File "/home/mike/opencodelists/codelists/models.py", line 449, in codeset
    return self._old_style_codeset()
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mike/opencodelists/codelists/models.py", line 460, in _old_style_codeset
    return Codeset.from_codes(set(self.codes), self.hierarchy)
                              ^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not iterable
2025-04-17T10:06:07.869865Z [error    ] Internal Server Error: /codelist/opensafely/asthma-inhaler-salbutamol-medication/2020-04-15/definition.csv [django.request] ip=127.0.0.1 request_id=e622272e-9ea5-4578-83b9-e299c5b3f047 user_id=mike
Traceback (most recent call last):
  File "/home/mike/opencodelists/.venv/lib/python3.12/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/home/mike/opencodelists/.venv/lib/python3.12/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mike/opencodelists/codelists/views/decorators.py", line 67, in wrapped_view
    rsp = view_fn(request, version, **view_kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mike/opencodelists/codelists/views/version_download_definition.py", line 13, in version_download_definition
    response.write(clv.definition_csv_data_for_download())
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mike/opencodelists/codelists/models.py", line 888, in definition_csv_data_for_download
    return rows_to_csv_data(present_definition_for_download(self))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mike/opencodelists/codelists/presenters.py", line 31, in present_definition_for_download
    codeset = clv.codeset
              ^^^^^^^^^^^
  File "/home/mike/opencodelists/codelists/models.py", line 449, in codeset
    return self._old_style_codeset()
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/mike/opencodelists/codelists/models.py", line 460, in _old_style_codeset
    return Codeset.from_codes(set(self.codes), self.hierarchy)
                              ^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not iterable

Defining delivery tasks guidance

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with codelists/views/version_download_definition.py and follow the call into codelists/models.py and codelists/presenters.py shown in the stack trace. Reproduce the unsupported coding-system URL, then verify that the view handles the error gracefully instead of returning a 500 response.

Written by the indexing model from the issue text.

Assessment

Tech stack
django, python
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.