Error 500 for corporate proxy

I ran into an error installing the sqlite3 node module where on one account it would install fine, while on another it doesn't. The difference turned out to be one using request, while the other, failing one, used needle. All use the same http_proxy URL from the environment.

node-pre-gyp WARN Using needle for node-pre-gyp https download
node-pre-gyp info check checked for "/tmp/webtest/node_modules/sqlite3/lib/binding/node-v64-linux-x64/node_sqlite3.node" (not found)
node-pre-gyp http GET https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v4.2.0/node-v64-linux-x64.tar.gz
node-pre-gyp verb download using proxy url: "http://giba-proxy.wps.ing.net:8090"
node-pre-gyp http 500 https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v4.2.0/node-v64-linux-x64.tar.gz
node-pre-gyp WARN Tried to download(500): https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v4.2.0/node-v64-linux-x64.tar.gz
node-pre-gyp WARN Pre-built binaries not found for sqlite3@4.2.0 and node@10.21.0 (node-v64 ABI, glibc) (falling back to source compile with node-gyp)
node-pre-gyp http 500 status code downloading tarball https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v4.2.0/node-v64-linux-x64.tar.gz

The account where it works shows this;

node-pre-gyp WARN Using request for node-pre-gyp https download
node-pre-gyp info check checked for "/appl/gmdb/mdbdev/home/qy69gz/work/bla/new/node_modules/sqlite3/lib/binding/node-v64-linux-x64/node_sqlite3.node" (not found)
node-pre-gyp http GET https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v4.2.0/node-v64-linux-x64.tar.gz
node-pre-gyp verb download using proxy url: "http://giba-proxy.wps.ing.net:8090/"
node-pre-gyp http 200 https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v4.2.0/node-v64-linux-x64.tar.gz
node-pre-gyp info install unpacking node-v64-linux-x64/node_sqlite3.node
node-pre-gyp info tarball done parsing tarball

I built a simple code example, using needle, request and https/https-proxy-agent.

const fs = require('fs');
const https = require("https");
const needle = require('needle');
const process = require('process');
const Proxy = require("https-proxy-agent");
const request = require('request');
const url = require("url");

const target = 'https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v4.2.0/node-v64-linux-x64.tar.gz';

var
  options,
  proxy,
  req;

// Raw https with https-proxy agent
options = url.parse(target);
options.agent = new Proxy(process.env.http_proxy);
req = https.get(options, response =>
{
  console.log(`https: statusCode: ${response && response.statusCode}`);
});
req.on("error", err =>
{
  console.log(`https: error in request; ${err.message}`);
});

// Deprecated request with forced proxy option
options = { proxy: process.env.http_proxy };
proxy = request.defaults(options);
proxy.get(target, options, (err, response, body) =>
{
  if (err)
  {
    console.log(`request: error in request; ${err.message}`);
    return;
  }

  console.log(`request statusCode: ${response && response.statusCode}`);
});

// Needle with proxy option
options = { proxy: process.env.http_proxy };
needle.get(target, options, (err, response, body) =>
{
  if (err)
  {
    console.log(`needle: error in request; ${err.message}`);
    return;
  }
 
  console.log(`needle: statusCode: ${response && response.statusCode}`);
});

The resulting HTTP codes are:

needle: statusCode: 500
https: statusCode: 200
request statusCode: 200

The same URL works fine in curl and wget, which both use the same http_proxy variable.

The proxy responds with the following:

Handshake failed
The SSL handshake could not be performed
Can't initialize server context:handshakefailed:server state 1:state 9:Application response 500 handshakefailed

Basically, this issue is preventing me from upgrading to node v10 as I need the binary part of the sqlite3 lib. Without that, I'm stuck on v6. The build fallback fails as this is an older version of the C-compiler that cannot be upgraded.

Start with the minimal JavaScript reproduction in the issue and compare Needle's proxy request with the working request and https-proxy-agent cases. Investigate why the corporate proxy returns HTTP 500 during the HTTPS handshake; done means the supplied S3 URL downloads successfully through the proxy using Needle.