Filebeat threat intel module integration - Doesn't get any data from MISP

Hello,

I've got the same problem using Filebeat threatintel module.

Here is my configuration from `filebeat.yml` (module enabled: AbuseURL, AbuseMalware and misp) :

```
# Module: threatintel
# Docs: https://www.elastic.co/guide/en/beats/filebeat/8.6/filebeat-module-threatintel.html

- module: threatintel
abuseurl:
enabled: true

# Input used for ingesting threat intel data.
var.input: httpjson

# The URL used for Threat Intel API calls.
var.url: https://urlhaus-api.abuse.ch/v1/urls/recent/

# The interval to poll the API for updates.
var.interval: 10m

abusemalware:
enabled: true

# Input used for ingesting threat intel data.
var.input: httpjson

# The URL used for Threat Intel API calls.
var.url: https://urlhaus-api.abuse.ch/v1/payloads/recent/

# The interval to poll the API for updates.
var.interval: 10m

malwarebazaar:
enabled: false

# Input used for ingesting threat intel data.
var.input: httpjson

# The URL used for Threat Intel API calls.
var.url: https://mb-api.abuse.ch/api/v1/

# The interval to poll the API for updates.
var.interval: 10m

misp:
enabled: true

# Input used for ingesting threat intel data, defaults to JSON.
var.input: httpjson

# The URL of the MISP instance, should end with "/events/restSearch".
var.url: https://localhost/events/restSearch

# The authentication token used to contact the MISP API. Found when looking at user account in the MISP UI.
var.api_token: BEpdSXuPb2lRyhVjNy9nHiA7EApYdD9ajMRafBZQ

# Configures the type of SSL verification done, if MISP is running on self signed certificates
# then the certificate would either need to be trusted, or verification_mode set to none.
var.ssl.verification_mode: none

# Optional filters that can be applied to the API for filtering out results. This should support the majority of fields in a MISP context.
# For examples please reference the filebeat module documentation.
#var.filters:
# - threat_level: [4, 5]
# - to_ids: true

# How far back to look once the beat starts up for the first time, the value has to be in hours. Each request afterwards will filter on any event newer
# than the last event that was already ingested.
var.first_interval: 1000h

# The interval to poll the API for updates.
var.interval: 5m

otx:
enabled: false
...
```
I well receive misp information in the index see the (http://localhost:9200/_search view):
```
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 10000,
"relation": "gte"
},
"max_score": 1,
"hits": [
{
"_index": ".ds-filebeat-8.6.2-2023.03.26-000001",
"_id": "FtxeWBNcKICyv7CHr2Kp0GR5r10=",
"_score": 1,
"_source": {
"input": {
"type": "httpjson"
},
"agent": {
"name": "nicop-IdeaPad-5-Pro-14ARH7",
"id": "12cc01ce-b83b-47d9-b318-a6c4eef588cd",
"type": "filebeat",
"ephemeral_id": "bf0c1db7-6458-4a77-941f-1cfd7c04d976",
"version": "8.6.2"
},
"@timestamp": "2023-03-16T00:03:09.000Z",
"ecs": {
"version": "1.12.0"
},
"service": {
"type": "threatintel"
},
"misp": {
"date": "2023-03-15",
"threat_level_id": 2,
"attribute_count": 5006,
"CryptographicKey": [],
"orgc": {
"name": "abuse.ch",
"id": "30",
"uuid": "9b086132-8588-49ed-97fd-8578a777822c",
"local": false
},
"distribution": "0",
"uuid": "72701075-2bf8-4b6b-9321-24cd65d77371",
"extends_uuid": "",
"proposal_email_lock": false,
"sharing_group_id": "0",
"context": {
"attribute": {
"distribution": 5,
"type": "url",
"object_id": "0",
"uuid": "2e6db5b3-c2f3-11ed-af1e-82aee05be84a",
"to_ids": true,
"disable_correlation": false,
"deleted": false,
"event_id": "7268",
"sharing_group_id": "0",
"comment": "Malware distribution site",
"Tag": [
{
"is_galaxy": false,
"colour": "#0b6b80",
"is_custom_galaxy": false,
"local_only": false,
"exportable": true,
"hide_tag": false,
"user_id": "0",
"name": "elf",
"id": "1418",
"local": 0
},
{
"is_galaxy": false,
"colour": "#dabae0",
"is_custom_galaxy": false,
"local_only": false,
"exportable": true,
"hide_tag": false,
"user_id": "0",
"name": "Mozi",
"id": "1420",
"local": 0
}
],
"id": "4047325",
"category": "Payload delivery",
"value": "http://222.137.167.12:42325/Mozi.m",
"timestamp": "1678858523"
}
},
"id": "7268",
"attribute": {
"distribution": 5,
"type": "md5",
"object_id": "490902",
"uuid": "c1378642-b8f3-4fec-9bfa-541f59b0fe43",
"object_relation": "md5",
"to_ids": true,
"disable_correlation": false,
"deleted": false,
"event_id": "7268",
"sharing_group_id": "0",
"comment": "Malware payload",
"Tag": [
{
"is_galaxy": false,
"colour": "#4f0baf",
"is_custom_galaxy": false,
"local_only": false,
"exportable": true,
"hide_tag": false,
"user_id": "0",
"name": "BB19",
"id": "2156",
"local": 0
},
{
"is_galaxy": false,
"colour": "#1f0be2",
"is_custom_galaxy": false,
"local_only": false,
"exportable": true,
"hide_tag": false,
"user_id": "0",
"name": "geofenced",
"id": "1476",
"local": 0
},
{
"is_galaxy": false,
"colour": "#bf35ee",
"is_custom_galaxy": false,
"local_only": false,
"exportable": true,
"hide_tag": false,
"user_id": "0",
"name": "js",
"id": "1446",
"local": 0
},
{
"is_galaxy": false,
"colour": "#f19758",
"is_custom_galaxy": false,
"local_only": false,
"exportable": true,
"hide_tag": false,
"user_id": "0",
"name": "Qakbot",
"id": "1481",
"local": 0
},
{
"is_galaxy": false,
"colour": "#c4b637",
"is_custom_galaxy": false,
"local_only": false,
"exportable": true,
"hide_tag": false,
"user_id": "0",
"name": "USA",
"id": "1477",
"local": 0
}
],
"id": "4050286",
"category": "Payload delivery",
"timestamp": "1678844586"
},
"locked": false,
"info": "URLhaus IOCs for 2023-03-15",
"published": true,
"orgc_id": "30",
"publish_timestamp": "0",
"disable_correlation": false,
"org_id": "1"
},
"threat": {
"indicator": {
"file": {
"hash": {
"md5": "77f6a87d88c2cf4544082e3016309b76"
}
},
"marking": {
"tlp": [
"white"
]
},
"provider": "misp",
"type": "file",
"scanner_stats": 1
},
"feed": {
"name": "[Filebeat] MISP",
"dashboard_id": "ad9c7430-72de-11eb-a3e3-b3cc7c78a70f"
}
},
"event": {
"ingested": "2023-03-26T18:55:23.829626133Z",
"created": "2023-03-26T18:55:20.483Z",
"timezone": "+02:00",
"kind": "enrichment",
"module": "threatintel",
"category": "threat",
"type": "indicator",
"dataset": "threatintel.misp"
},
"fileset": {
"name": "misp"
},
"user": {
"roles": [
"reporting_user"
],
"email": "admin@admin.test"
},
"tags": [
"type:OSINT",
"tlp:white"
]
}
},
{
"_index": ".ds-filebeat-8.6.2-2023.03.26-000001",
"_id": "iplrKQys9BW32sHn3iQYkMJ2DlY=",
"_score": 1,
"_source": {
"input": {
"type": "httpjson"
},
"agent": {
"name": "nicop-IdeaPad-5-Pro-14ARH7",
"id": "12cc01ce-b83b-47d9-b318-a6c4eef588cd",
"type": "filebeat",
"ephemeral_id": "bf0c1db7-6458-4a77-941f-1cfd7c04d976",
"version": "8.6.2"
},
"@timestamp": "2023-03-16T00:03:09.000Z",
"ecs": {
"version": "1.12.0"
},
"service": {
"type": "threatintel"
},
"misp": {
"date": "2023-03-15",
"threat_level_id": 2,
"attribute_count": 5006,
"CryptographicKey": [],
"orgc": {
"name": "abuse.ch",
"id": "30",
"uuid": "9b086132-8588-49ed-97fd-8578a777822c",
"local": false
},
"distribution": "0",
"uuid": "72701075-2bf8-4b6b-9321-24cd65d77371",
"extends_uuid": "",
"proposal_email_lock": false,
"sharing_group_id": "0",
"context": {
"attribute": {
"distribution": 5,
"type": "url",
"object_id": "0",
"uuid": "2e6db5b3-c2f3-11ed-af1e-82aee05be84a",
"to_ids": true,
"disable_correlation": false,
"deleted": false,
"event_id": "7268",
"sharing_group_id": "0",
"comment": "Malware distribution site",
"Tag": [
{
"is_galaxy": false,
"colour": "#0b6b80",
"is_custom_galaxy": false,
"local_only": false,
"exportable": true,
"hide_tag": false,
"user_id": "0",
"name": "elf",
"id": "1418",
"local": 0
},
{
"is_galaxy": false,
"colour": "#dabae0",
"is_custom_galaxy": false,
"local_only": false,
"exportable": true,
"hide_tag": false,
"user_id": "0",
"name": "Mozi",
"id": "1420",
"local": 0
}
],
```

Here is the logs that i receive on Kibana (only Abuse.sh logs):

Here is one link to elastic discuss forum:
https://discuss.elastic.co/t/bug-no-misp-event-data-send-to-kibana-when-threat-intel-module-used/327979
https://discuss.elastic.co/t/threat-intel-filebeat-module-i-dont-get-any-data-from-misp-and-i-dont-know-why/327986.