TheHive-Project / TheHive-Project/Cortex
issue running cortexneurons via docker
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 264
- PR merge metrics
- No merged PRs in 30d
Description
Request Type
Bug
Work Environment
| Question | Answer |
|---|---|
| OS version (server) | Ubuntu |
| Cortex version / git hash | docker image: 'thehiveproject/cortex:3.1.1-1'
| Package Type | Docker
Problem Description
I've setup cortex on a dedicated server.
Cortex run as a docker using docker compose, using instruction at
https://docs.strangebee.com/cortex/installation-and-configuration/run-cortex-with-docker/#use-docker-compose
yml file is the following:
version: "2"
services:
elasticsearch:
image: elasticsearch:7.11.1
environment:
- http.host=0.0.0.0
- discovery.type=single-node
- script.allowed_types=inline
- thread_pool.search.queue_size=100000
- thread_pool.write.queue_size=10000
cortex:
image: 'thehiveproject/cortex:3.1.1-1'
environment:
- 'job_directory=${job_directory}'
volumes:
- '/var/run/docker.sock:/var/run/docker.sock'
- '${job_directory}:${job_directory}'
depends_on:
- elasticsearch
ports:
- '0.0.0.0:9001:9001'
I can start the service with docker-compose up, I can finalize the configuration, setup few analyzers with required API
but when I try to run any of the analyzers I get the following error:
Request error: POST unix://localhost:80/images/create?fromImage=docker.io%2Fcortexneurons%2Fvirustotal_getreport&tag=3: 500, body: {"message":"toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"}
or
Request error: POST unix://localhost:80/images/create?fromImage=docker.io%2Fcortexneurons%2Fabuseipdb&tag=1: 500, body: {"message":"toomanyrequests: You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit"}
even if I successfully manually login to docker.com with the proper user/token the error persist, as if the further download of the cortexneurons (initial pull) is not using such authentication and fails because of rate-limit on the docker.com side.
-
Sometimes, when I do not hit the rate limit threshold, the analyzer works as expected.
-
Is there a way to configure authentication parameters to be used whenever a pull is required?
Log when error occurs:
cortex-1 | [info] o.t.c.s.JobSrv - Looking for similar job in the last 10 minutes (worker=3e67614b9a076dba5ed4aa7fd53c75b7, dataType=ip, data=Left(8.8.8.8), tlp=2, parameters={})
cortex-1 | [info] o.t.c.s.AuditActor - Job 0e6X_JIBzEDw4VX1_5dT has be updated (JsDefined("InProgress"))
cortex-1 | [info] o.t.c.s.AuditActor - Job 0e6X_JIBzEDw4VX1_5dT has be updated (JsDefined("Failure"))
cortex-1 | [info] o.t.c.s.JobSrv - Job 0e6X_JIBzEDw4VX1_5dT has finished with status Failure
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the linked Docker Compose installation instructions and the supplied compose YAML, especially the Cortex container's Docker socket and analyzer image pulls. Reproduce an analyzer run after Docker authentication, then determine whether authenticated credentials are passed to the daemon; done means analyzer images can be pulled without Docker Hub rate-limit failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, docker-compose
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100