Dockerized Cortex cannot reach out for default analyzers and responders list, when running behind proxy

Request Type

Bug

Work Environment

Problem Description

I have a Cortex instance configured to use the default list of workers along with a custom set of my own.
The default set is pulled from https://thehive-project.org, along the lines of the examples seen at TheHiveProject github.

The relevant excerpt from my application.conf:

analyzer {
  urls = [
    "https://download.thehive-project.org/analyzers.json",
    "/opt/cortex/custom-analyzers"
  ]
}

responder {
  urls = [
    "https://download.thehive-project.org/responders.json",
    "/opt/cortex/custom-responders"
  ]
}

and from docker-compose.yaml, for more complete perspective:

  cortex:
    command: --job-directory /opt/cortex/jobs
    container_name: hcecpn_cortex
    depends_on:
    - elasticsearch
    environment:
      JOB_DIRECTORY: /opt/cortex/jobs
    image: thehiveproject/cortex:3.1.3
    ports:
    - '0.0.0.0:9001:9001'
    volumes:
    - /home/user/hive4test/application.conf:/etc/cortex/application.conf:ro
    - /home/user/hive4test/Cortex_Workers/analyzers:/opt/cortex/custom-analyzers:ro
    - /home/user/hive4test/Cortex_Workers/responders:/opt/cortex/custom-responders:ro
    - /home/user/hive4test/vol/cortex/jobs:/opt/cortex/jobs:rw
    - /var/run/docker.sock:/var/run/docker.sock:rw

When I create and start the container, and later log in to the application via its web interface, the only analyzers and responders I see are my custom ones. Cortex seems to not be able to pull and display the default set.

Also, the following shows up in the application log:

[warn] o.t.c.s.WorkerSrv - Worker path (https://download.thehive-project.org/analyzers.json) is not found
[warn] o.t.c.s.WorkerSrv - Worker path (https://download.thehive-project.org/responders.json) is not found

The thing is I'm running it behind a proxy, and my guess is that it blocks the attempt to pull the default workers list upon app startup.

My questions:

1. Is my guess correct and the proxy is indeed the problem?
2. If so, how can I instruct the Cortex app to use proxy for start up? (or to whatever is necessary for it to see these locations)

Complementary information

So far I tried to type the proxy info by hand into the docker-compose.yaml file like this:

    environment:
      - no_proxy=...
      - http_proxy=...
      - https_proxy=...
      - NO_PROXY=...
      - HTTP_PROXY=...
      - HTTPS_PROXY=...

but basically to no avail. With these modifications the environment inside the container was aware of the proxy and I was able to eg. wget by hand the problematic JSON files from the shell in there after I started the container, but even then the Cortex application as such was still not able to reach the files upon it's startup.

When Cortex is up and running, there is a nice and neat way to enter the proxy details for the workers via the web interface, but is there any way to inform the application itself about the proxy?

Start with the WorkerSrv warning and the analyzer/responder URL handling, using the application.conf and docker-compose.yaml excerpts as the reproduction setup. Check how Cortex startup handles proxy settings when fetching remote worker lists, then reproduce behind a proxy. Done means the default analyzers.json and responders.json lists load alongside the custom workers.