Retries on 503/504 errors

Proposal:

Hi! In some situations, Manticore nodes might be running behind another service (i.e. load balancer, proxy, firewall, etc) and we have noticed that if there are timeouts between the proxy and manticore, we see a 503 or 504 in the client.

However, the PHP client doesn't seem to handle those scenarios. It simply returns "fatal error while trying to decode JSON response" or "Manticoresearch\Exceptions\RuntimeException".

There are some things that could be improved IMHO. a) reporting of the exception, it should properly say it was a 503 or 504 and b) the client should have the ability to retry these if needed.

For now, we have implemented a workaround for both issues (i.e. retrying on exceptions of the PHP client) and adding this to the client config:

'curl' => self::curlResponseCapture(),
.
.
.
    // The manticoresearch-php library uses curl to make the requests
    // but doesn't tell us the HTTP status code or response headers which
    // are useful for troubleshooting. So adding this custom function to capture them.
    private static function curlResponseCapture() {
        return [
            CURLOPT_HEADERFUNCTION => function($ch, $header) {
                if (preg_match('/^HTTP\/[\d.]+\s+(\d+)/', $header, $matches)) {
                    self::$lastHttpStatus = (int)$matches[1];
                    self::$lastResponseHeaders = [];
                }
                $trimmed = trim($header);
                if ($trimmed !== '') {
                    self::$lastResponseHeaders[] = $trimmed;
                }
                return strlen($header);
            },
            CURLOPT_WRITEFUNCTION => function($ch, $data) {
                self::$lastResponseBody = $data;
                echo $data;
                return strlen($data);
            },
        ];
    }

Checklist:

^{To be completed by the assignee. Check off tasks that have been completed or are not applicable.}