impossible to search job with observable field

Impossible to search job with observable field

Request Bug

Work Environment

Problem Description

In the job panel, the search of jobs is impossible with observable field not empty

Steps to Reproduce

1. Fill the observable field
2. Click to search
3. Nothing...

Complementary information

This exception appears in the LOGS with the 8.8.8.8 value in observale :

cortex    | [error] o.e.d.DBConfiguration - ElasticSearch request failure: POST:/cortex_4/_search?
cortex    | StringEntity({"version":"true","query":{"bool":{"must":[{"term":{"relations":{"value":"job"}}},{"bool":{"must":[{"term":{"organization":{"value":"local"}}},{"bool":{"must":[{"term":{"data":{"value":"8.8.8.8"}}},{"bool":{"must_not":[{"term":{"status":{"value":"Deleted"}}}]}}]}}]}}]}},"from":0,"size":10,"sort":[{"createdAt":{"order":"desc"}},{"_id":{"order":"desc"}}]},Some(application/json))
cortex    |  => ElasticError(search_phase_execution_exception,all shards failed,None,None,None,List(ElasticError(query_shard_exception,Binary fields do not support searching,Some(ROiuxZqqStODqmwWsQrMzw),Some(cortex_4),None,null,None)),None)
cortex    | [info] o.t.c.s.ErrorHandler - POST /api/job/_search?range=0-10&sort=-createdAt returned 400
cortex    | org.elastic4play.SearchError: all shards failed
cortex    | 	at org.elastic4play.database.DBConfiguration.$anonfun$execute$2(DBConfiguration.scala:147)
cortex    | 	at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)
cortex    | 	at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)
cortex    | 	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
cortex    | 	at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
cortex    | 	at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)
cortex    | 	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
cortex    | 	at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:85)
cortex    | 	at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91)
cortex    | 	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)

A sample CURL request with the observale field (data) show the problem :

# curl --request GET   --url http://localhost:9200/cortex_4/_search   --header 'content-type: application/json'   --data '{
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "data": {
              "value": "8.8.4.4"
            }
          }
        }
      ]
    }
  }
}'
{"error":{"root_cause":[{"type":"query_shard_exception","reason":"Binary fields do not support searching","index_uuid":"1FBCuimpRlKwWwTCyOSv7Q","index":"cortex_4"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"cortex_4","node":"1u5gKO9VSAy38DPA4cwRVg","reason":{"type":"query_shard_exception","reason":"Binary fields do not support searching","index_uuid":"1FBCuimpRlKwWwTCyOSv7Q","index":"cortex_4"}}]},"status":400}

Without the field observale (data) the search is OK :

# curl --request GET   --url http://localhost:9200/cortex_4/_search   --header 'content-type: application/json'   --data '{
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "relations": {
              "value": "job"
            }
          }
        }
      ]
    }
  }
}'
{"took":35,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":2,"max_score":1.6739764,"hits":[{"_index":"cortex_4","_type":"doc","_id":"U6vJKnABbLIwunFoOeSI","_score":1.6739764,"_source":{"workerId":"fbf736028bb400859eef20f44d96f708","data":"8.8.4.4","dataType":"ip","type":"analyzer","message":"","cacheTag":"6f27554ec11aa04402f2c78070ce471e","createdAt":1581265795420,"createdBy":"ghislain-bernard","organization":"local","tlp":1,"workerDefinitionId":"template_0_0_1","workerName":"template_0_0_1","relations":"job","parameters":"{}","pap":1,"status":"Success","updatedBy":"ghislain-bernard","startDate":1581265796290,"updatedAt":1581265862412,"endDate":1581265862410}},{"_index":"cortex_4","_type":"doc","_id":"VKvJKnABbLIwunFoc-T0","_score":0.2876821,"_source":{"workerId":"fbf736028bb400859eef20f44d96f708","data":"8.8.8.8","dataType":"ip","type":"analyzer","message":"","cacheTag":"46c9360294f3402d9e6788d171f2a6de","createdAt":1581265810387,"createdBy":"ghislain-bernard","organization":"local","tlp":1,"workerDefinitionId":"template_0_0_1","workerName":"template_0_0_1","relations":"job","parameters":"{}","pap":1,"status":"InProgress","updatedBy":"ghislain-bernard","startDate":1581265811270,"updatedAt":1581265811274}}]}}

Start with the POST /api/job/_search endpoint and the Elasticsearch query shown in the issue, then trace how the observable data field is mapped and searched. Reproduce the failure with the supplied curl request and confirm that job searches using an observable value complete successfully without the Binary fields do not support searching error.