RemoveToxic: HTTP 503: Unexpected response code, expected 204

Sometimes (not always) I receive this error.

This is an example from my log:

...
gengw_mdt Added upstream reset_peer toxic 'reset_peer_upstream' on proxy 'gengw_mdt'
gengw_mdt Added upstream timeout toxic 'timeout_upstream' on proxy 'gengw_mdt'
gengw_mdt Removed toxic 'timeout_upstream' on proxy 'gengw_mdt'
gengw_mdt Removed toxic 'reset_peer_upstream' on proxy 'gengw_mdt'
gengw_mdt: going to sleep for 394 seconds
gengw_mdt: adding reset_peer toxicity
gengw_mdt Added downstream reset_peer toxic 'reset_peer_downstream' on proxy 'gengw_mdt'
gengw_mdt Removed toxic 'reset_peer_downstream' on proxy 'gengw_mdt'
gengw_mdt: going to sleep for 459 seconds
gengw_mdt: adding bandwidth_50kb toxicity for 246 seconds
gengw_mdt Added downstream bandwidth toxic 'bandwidth_downstream' on proxy 'gengw_mdt'
Failed to remove toxic: failed to remove toxic 'bandwidth_downstream' from proxy 'gengw_mdt': RemoveToxic: HTTP 503: Unexpected response code, expected 204
gengw_mdt
gengw_mdt: going to sleep for 512 seconds
gengw_mdt: adding reset_peer toxicity
Failed to add toxic: failed to retrieve proxy with name `gengw_mdt`: Proxy: HTTP 503: Unexpected response code, expected 200
gengw_mdt
Failed to remove toxic: failed to retrieve proxy with name `gengw_mdt`: Proxy: HTTP 503: Unexpected response code, expected 200
gengw_mdt
gengw_mdt: going to sleep for 338 seconds
gengw_mdt: adding bandwidth_50kb toxicity for 170 seconds
Failed to add toxic: failed to retrieve proxy with name `gengw_mdt`: Proxy: HTTP 503: Unexpected response code, expected 200
gengw_mdt
Failed to remove toxic: failed to retrieve proxy with name `gengw_mdt`: Proxy: HTTP 503: Unexpected response code, expected 200
gengw_mdt

Start from the reported RemoveToxic failure and the accompanying proxy log for gengw_mdt; investigate how intermittent HTTP 503 responses are handled when removing or retrieving a toxic. Done means reproducing or explaining the unexpected response and adding a verified fix or regression coverage.