elastic / elastic/integrations

[Suricata]: field suricata.eve.stats.flow.end is incorrecly assumed as date

Open
#16,331 2 comments 0 reactions 0 assignees View on GitHub
Integration:suricata needs:triage Team:Integration-Experience
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
3d 4h
Merged PRs (30d)
209

Description

### Integration Name

Suricata [suricata]

### Dataset Name

suricata.eve

### Integration Version

2.27.0

### Agent Version

8.19.6

### Agent Output Type

logstash

### Elasticsearch Version

8.19.6

### OS Version and Architecture

Debian 12.12 (x86_64)

### Software/API Version

Suricata 7.0.8

### Error Message

Error message was: Elastichsearch was expecting the field suricata.eve.stats.flow.end to be of type 'date', but instead it was 'object'

### Event Original

No actual log available, but it's the suricata json-format

### What did you do?

Integration configured with defaults, only path of the logfile was changed. In suricata, the 'stats' is enabled.

### What did you see?

Events from logstash to elasticsearch were dropped into dead letter queue if they included the "stats" -block.

### What did you expect to see?

Events in the datastream

### Anything else?

After some debugging, there seems to be faulty mapping in the component template `logs-suricata.eve@package`. The problem is this block:
```
{
"_embedded_ecs-end_to_date": {
"mapping": {
"type": "date"
},
"match": "end"
}
},
```

But actually it probably is inherited from `ecs@mappings` template's `ecs_date` -block;
```
"ecs_date": {
"path_match": [
"*.timestamp",
"*_timestamp",
"*.not_after",
"*.not_before",
"*.accessed",
"created",
"*.created",
"*.installed",
"*.creation_date",
"*.ctime",
"*.mtime",
"ingested",
"*.ingested",
"*.start",
"*.end",
"*.indicator.first_seen",
"*.indicator.last_seen",
"*.indicator.modified_at",
"*threat.enrichments.matched.occurred"
],
"mapping": {
"type": "date"
},
"unmatch_mapping_type": "object"
}
```

Which matches (?) all fields named 'end' and tries to map those as type `date`. But the field `suricata.eve.stats.flow.end` is not in fact a date, but an object, as per: https://docs.suricata.io/en/suricata-8.0.2/appendix/eve-index.html#stats-flow-end-object

To summarize: all fields named 'end' are not timestamps and especially not in Suricata -context.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.