TheHive-Project / TheHive-Project/Cortex
[BUG] Cortex is unresponsive if too much jobs
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 264
- PR merge metrics
- No merged PRs in 30d
Description
Request Type
Bug
Work Environment
| Question | Answer |
|---|---|
| OS version (server) | Docker SWARM |
| OS version (client) | Viperr, |
| Virtualized Env. | True |
| Dedicated RAM | 16 GB |
| vCPU | 8 |
| Cortex version / git hash | 3.1.1 |
| Package Type | RPM, DEB, Docker, Binary, From source |
| Index type | Elasticsearch |
| Attachments storage | Local (GlusterFS) |
| Browser type & version | Firefox and Chrome |
Problem Description
When ending a big quantity of artefact to Cortex to get analyze by a few analyzers, cortex becam unresponsive. Front Page is blank while answering code 200 and it is impossible to get access or communication using API. At the end of all jobs that continue running, service is again available.
Issue is report are not sent back to thehive, you need to rerun analyzer and result is given directly (cached result)
Steps to Reproduce
- Add some artifact
- Run them to a big quantity of Analyzer
- Observate the unresponsivness
Possible Solutions
I did add this to the application.conf this helped in some case but not all.
akka {
log-config-on-start = on
actor {
default-dispatcher {
fork-join-executor {
parallelism-max = 16
}
thread-pool-executor {
fixed-pool-size = 16
}
throughput = 1
}
default-blocking-io-dispatcher {
fork-join-executor {
parallelism-max = 32
}
thread-pool-executor {
fixed-pool-size = 32
}
throughput = 1
}
}
}
Complementary information
(add anything that can help identifying the problem such as log excerpts, screenshots, configuration dumps etc.)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the failure with a large batch of analyzer jobs and inspect the service logs alongside application.conf. Trace the analyzer-job processing and API/front-page availability while jobs run, including whether reports are returned to TheHive. Done means Cortex remains responsive under the reproduced load and reports are delivered without requiring a rerun.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, elasticsearch, scala
- Domain
- api, backend, performance
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100