TheHive-Project / TheHive-Project/Cortex

cortex does not correctly start and brings strange warning messages about docker even it is not installed

Open
#268 10 comments 1 reaction 1 assignee View on GitHub

@To-om is already working on this.

Since Jun 3, 2020.

Dominant language
Scala
Stars
1.6k
Forks
264
PR merge metrics
No merged PRs in 30d

Description

Request Type

Bug

Work Environment
Question Answer
OS version (server) RHEL 7
OS version (client) 10
Cortex version / git hash 3.0.1
Package Type Binary
Problem Description

Cortex failes to start Analyzers (e.g. MISP) with strange error messages.
In this setup there is no docker installed and everything comes from local files (classic-mode).

There are some strange warnings about missing cortexutils for python/python3, but it is installed for both python versions.

Is it required to install docker to use Cortex with Cortex 3?

Complementary information

If I directly start cortex as cortex user (for testing purpose only).
/opt/cortex/bin/cortex -Dconfig.file=/etc/cortex/application.conf -Dlogger.file=/etc/cortex/logback.xml
I can see the following strange errors:

[info] o.r.Reflections - Reflections took 124 ms to scan 2 urls, producing 99 keys and 979 values
[info] module - Loading model class org.thp.cortex.models.OrganizationModel
[info] module - Loading model class org.thp.cortex.models.ArtifactModel
[info] module - Loading model class org.thp.cortex.models.WorkerConfigModel
[info] module - Loading model class org.elastic4play.services.DBListModel
[info] module - Loading model class org.thp.cortex.models.ReportModel
[info] module - Loading model class org.elastic4play.services.AttachmentModel
[info] module - Loading model class org.thp.cortex.models.WorkerModel
[info] module - Loading model class org.thp.cortex.models.JobModel
[info] module - Loading model class org.thp.cortex.models.UserModel
[info] module - Loading model class org.thp.cortex.models.AuditModel
[info] module - Loading authentication module class org.thp.cortex.services.LocalAuthSrv
[info] module - Loading authentication module class org.elastic4play.services.auth.LdapAuthSrv
[info] module - Loading authentication module class org.elastic4play.services.auth.ADAuthSrv
[info] module - Loading authentication module class org.thp.cortex.services.KeyAuthSrv
[info] module - Loading authentication module class org.thp.cortex.services.OAuth2Srv
[info] a.e.s.Slf4jLogger - Slf4jLogger started
[info] c.s.e.h.ElasticClient$ - Creating HTTP client on http://127.0.0.1:9200
[warn] application - /etc/cortex/application.conf: 144: analyzer.path is deprecated, use analyzer.urls instead
[info] o.a.h.i.e.RetryExec - I/O exception (java.io.IOException) caught when processing request to {}->unix://localhost:80: No such file or directory
[info] o.a.h.i.e.RetryExec - Retrying request to {}->unix://localhost:80
[info] o.a.h.i.e.RetryExec - I/O exception (java.io.IOException) caught when processing request to {}->unix://localhost:80: No such file or directory
[info] o.a.h.i.e.RetryExec - Retrying request to {}->unix://localhost:80
[info] o.a.h.i.e.RetryExec - I/O exception (java.io.IOException) caught when processing request to {}->unix://localhost:80: No such file or directory
[info] o.a.h.i.e.RetryExec - Retrying request to {}->unix://localhost:80
[info] o.t.c.s.DockerJobRunnerSrv - Docker is not available
com.spotify.docker.client.exceptions.DockerException: java.util.concurrent.ExecutionException: javax.ws.rs.ProcessingException: java.io.IOException: No such file or directory
        at com.spotify.docker.client.DefaultDockerClient.propagate(DefaultDockerClient.java:2828)
        at com.spotify.docker.client.DefaultDockerClient.request(DefaultDockerClient.java:2692)
        at com.spotify.docker.client.DefaultDockerClient.info(DefaultDockerClient.java:595)
        at org.thp.cortex.services.DockerJobRunnerSrv.$anonfun$isAvailable$2(DockerJobRunnerSrv.scala:47)
        at play.api.LoggerLike.info(Logger.scala:160)
        at play.api.LoggerLike.info$(Logger.scala:157)
        at play.api.Logger.info(Logger.scala:251)
        at org.thp.cortex.services.DockerJobRunnerSrv.$anonfun$isAvailable$1(DockerJobRunnerSrv.scala:47)
        at scala.runtime.java8.JFunction0$mcZ$sp.apply(JFunction0$mcZ$sp.java:23)
        at scala.util.Try$.apply(Try.scala:213)
Caused by: java.util.concurrent.ExecutionException: javax.ws.rs.ProcessingException: java.io.IOException: No such file or directory
        at jersey.repackaged.com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
        at jersey.repackaged.com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
        at jersey.repackaged.com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
        at com.spotify.docker.client.DefaultDockerClient.request(DefaultDockerClient.java:2690)
        at com.spotify.docker.client.DefaultDockerClient.info(DefaultDockerClient.java:595)
        at org.thp.cortex.services.DockerJobRunnerSrv.$anonfun$isAvailable$2(DockerJobRunnerSrv.scala:47)
        at play.api.LoggerLike.info(Logger.scala:160)
        at play.api.LoggerLike.info$(Logger.scala:157)
        at play.api.Logger.info(Logger.scala:251)
        at org.thp.cortex.services.DockerJobRunnerSrv.$anonfun$isAvailable$1(DockerJobRunnerSrv.scala:47)
Caused by: javax.ws.rs.ProcessingException: java.io.IOException: No such file or directory
        at org.glassfish.jersey.apache.connector.ApacheConnector.apply(ApacheConnector.java:481)
        at org.glassfish.jersey.apache.connector.ApacheConnector$1.run(ApacheConnector.java:491)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at jersey.repackaged.com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
        at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
        at jersey.repackaged.com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:50)
        at jersey.repackaged.com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:37)
        at org.glassfish.jersey.apache.connector.ApacheConnector.apply(ApacheConnector.java:487)
        at org.glassfish.jersey.client.ClientRuntime$2.run(ClientRuntime.java:178)
Caused by: java.io.IOException: No such file or directory
        at jnr.unixsocket.UnixSocketChannel.doConnect(UnixSocketChannel.java:127)
        at jnr.unixsocket.UnixSocketChannel.connect(UnixSocketChannel.java:136)
        at jnr.unixsocket.UnixSocketChannel.connect(UnixSocketChannel.java:223)
        at com.spotify.docker.client.UnixConnectionSocketFactory.connectSocket(UnixConnectionSocketFactory.java:85)
        at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:141)
        at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
        at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
Traceback (most recent call last):
  File "/usr/local/bin/pip", line 7, in <module>
    from pip._internal.cli.main import main
ModuleNotFoundError: No module named 'pip._internal'
Traceback (most recent call last):
  File "/usr/local/bin/pip", line 7, in <module>
    from pip._internal.cli.main import main
ModuleNotFoundError: No module named 'pip._internal'
[warn] o.t.c.s.JobRunnerSrv - The package cortexutils for python hasn't been found
[warn] o.t.c.s.JobRunnerSrv - The package cortexutils for python hasn't been found
Traceback (most recent call last):
  File "/bin/pip2", line 7, in <module>
    from pip._internal.cli.main import main
ImportError: No module named pip._internal.cli.main
[warn] o.t.c.s.JobRunnerSrv - The package cortexutils for python2 hasn't been found
Traceback (most recent call last):
  File "/bin/pip2", line 7, in <module>
    from pip._internal.cli.main import main
ImportError: No module named pip._internal.cli.main
[warn] o.t.c.s.JobRunnerSrv - The package cortexutils for python2 hasn't been found
Traceback (most recent call last):
  File "/usr/local/bin/pip3", line 7, in <module>
    from pip._internal.cli.main import main
ModuleNotFoundError: No module named 'pip._internal'
[warn] o.t.c.s.JobRunnerSrv - The package cortexutils for python3 hasn't been found
Traceback (most recent call last):
  File "/usr/local/bin/pip3", line 7, in <module>
    from pip._internal.cli.main import main
ModuleNotFoundError: No module named 'pip._internal'
[warn] o.t.c.s.JobRunnerSrv - The package cortexutils for python3 hasn't been found
[info] o.t.c.s.WorkerSrv - New worker list:

        IPVoid 1.0
        HIBP_Query 2.0
        DNSSinkhole 1.0
        Cyberprotect_ThreatScore 1.0
        Autofocus_SearchJSON 1.0
        DomainTools_Reputation 2.0
        [...]
        Mnemonic_pDNS_Closed 3.0
        UnshortenLink 1.2

[info] play.api.Play - Application started (Prod)
[info] p.c.s.AkkaHttpServer - Enabling HTTP/2 on Akka HTTP server...
[info] p.c.s.AkkaHttpServer - Listening for HTTP on /0.0.0.0:9001
^[[5~ ^C[info] p.c.s.AkkaHttpServer - Stopping server...

Cortex is started, and correctly working, never the less systemd seems also have a strange status:

systemctl status cortex
* cortex.service - cortex
   Loaded: loaded (/etc/systemd/system/cortex.service; enabled; vendor preset: disabled)
   Active: failed (Result: exit-code) since Mon 2020-05-04 08:28:17 CDT; 17min ago
     Docs: https://thehive-project.org
  Process: 89796 ExecStart=/opt/cortex/bin/cortex -Dconfig.file=/etc/cortex/application.conf -Dlogger.file=/etc/cortex/logback.xml -Dpidfile.path=/dev/null (code=exited, status=255)
 Main PID: 89796 (code=exited, status=255)

May 04 08:28:09 hostname systemd[1]: Started cortex.
May 04 08:28:17 hostname systemd[1]: cortex.service: main process exited, code=exited, status=255/n/a
May 04 08:28:17 hostname systemd[1]: Unit cortex.service entered failed state.
May 04 08:28:17 hostname systemd[1]: cortex.service failed.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.