Monarch condition text search - JSON error doesn't identify Monarch as the cause
- Dominant language
- Python
- Stars
- 30
- Forks
- 3
- Avg merge
- 9h 28m
- Merged PRs (30d)
- 42
Description
🤖 Written by Claude
Condition text matching hits the Monarch search API in `classification/models/condition_text_search.py`. When Monarch returns something that isn't JSON, `.json()` raises and the error that reaches Rollbar looks like this:
```
JSONDecodeError Expecting value: line 2 column 1 (char 1)
at search_suggestion (classification/models/condition_text_matching.py:995)
at condition_text_search (classification/models/condition_text_search.py:18)
at json (requests/models.py:1120)
```
Nothing in that says "Monarch", or what Monarch actually sent back. Each time it lands it reads as a JSON parsing bug in our code, and has to be re-diagnosed from the frame's `args` — this is the same failure as Rollbar item 7269 / #1603, which was investigated once already.
Propagating the failure is the behaviour we want — see #1603, where a `try/except` that swallowed these was deliberately reverted in 8c6c45aad, because returning `[]` made a Monarch outage indistinguishable from a genuine no-match and suppressed the upstream `report_exc_info()`. This issue is only about making the propagated error name its cause.
## Why the status code is missing
That revert also dropped the `raise_for_status()` that had come with the guard. So a 502 HTML error page from Monarch's gateway reaches `.json()` as a 200-looking response and dies on the parse, with no status code anywhere in the report.
## Not a specific condition text
The live API shrugs off Solr metacharacters (`"`, `[`, `(`, `\`, `^`, `~`, `a:b`, bare `AND`/`OR`) and query texts up to 10k chars — all return valid JSON 200s. So this is a transient upstream failure rather than a poison-pill search text, which is also why a single retry is worth having.
## Proposed
- `raise_for_status()` before `.json()`, so an upstream 5xx reports as an HTTP error naming the status and the Monarch URL.
- Wrap the `.json()` failure in an error that names Monarch and includes the content type and the start of the body, for the case where Monarch returns 200 with a non-JSON body.
- Retry once on 5xx, following the `urllib3` `Retry` + mounted-session pattern already in `library/utils/misc_utils.py`, so a single blip doesn't produce a Rollbar item per condition text processed.
Failures still propagate to the existing `report_exc_info()` handlers in `condition_text_matching.search_suggestion` and `ontology_matching`; only the message changes.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with classification/models/condition_text_search.py and the search_suggestion path in classification/models/condition_text_matching.py, then compare the urllib3 Retry and mounted-session pattern in library/utils/misc_utils.py. Verify the existing report_exc_info() handlers in search_suggestion and ontology_matching. Done means propagated Monarch failures identify the upstream status or non-JSON response and a single transient 5xx is retried without swallowing the error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100