elastic / elastic/integrations
[Zeek] Fix zeek.smtp ingest pipeline processor order and missing date formats
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 209
Description
### Integration Name
Zeek [zeek]
### Dataset Name
zeek.smtp
### Integration Version
4.0.0
### Agent Version
9.2.2
### Agent Output Type
logstash
### Elasticsearch Version
9.2.2
### OS Version and Architecture
Ubuntu 24.04
### Software/API Version
_No response_
### Error Message
Experiencing this error when trying to intest smtp zeek logs.
Inspecting the pipeline I see that this
```
{
"set": {
"tag": "set_event_created_e3f09e3b",
"field": "event.created",
"copy_from": "@timestamp"
}
}
```
is **before** the zeek.smtp.ts field is parsed into @timestamp, i.e., before this processor
```
{
"date": {
"tag": "date_zeek_smtp_ts_ae12ab0f",
"field": "zeek.smtp.ts",
"formats": [
"UNIX",
"ISO8601"
]
}
}
```
and therefore, experiencing this error
`"Processor 'set' with tag 'set_event_created_e3f09e3b' in pipeline '_simulate_pipeline' failed with message 'field [@timestamp] not present as part of path [@timestamp]'"
`
Putting the date processor before the set_event_created_e3f09e3b solves the problem.
Also I have detected a problem with this
` "Processor 'conditional' with tag 'date_zeek_smtp_date_to_zeek_smtp_date_71fb0b24' in pipeline '_simulate_pipeline' failed with message 'Text 'Tue Feb 10 06:26:42 AM CET 2026' could not be parsed at index 3'"`
As you can see the date format does not match I neede to add `EEE MMM d HH:mm:ss a z yyyy` and `EEE, d MMM yyyy HH:mm:ss Z (z)` to this date processor `date_zeek_smtp_date_to_zeek_smtp_date_71fb0b24`
### Event Original
`{"ts":1770701036.331145,"uid":"C9w5hFT8l22x2xWMd","id.orig_h":"192.168.9.10","id.orig_p":35006,"id.resp_h":"192.168.5.12","id.resp_p":25,"trans_depth":1,"helo":"mail.redacted.xy","mailfrom":"sonda@redacted.xy","rcptto":["incoming@srvmailarchive"],"date":"Tue Feb 10 06:26:42 AM CET 2026","from":"\"Sonda\" ","to":["\"\" "],"subject":"Test message Tue Feb 10 05:26:42 AM Europa 2026","first_received":"from redacted.xy (unknown [192.168.5.2])\tby mail.redacted.xy (Postfix) with SMTP id 28A4A3B\tfor ; Tue, 10 Feb 2026 06:26:40 +0100 (CET)","last_reply":"250 2.0.0 Ok: queued as 83AC31CCA9EB","path":["192.168.5.12","192.168.9.10","192.168.5.2"],"tls":false,"fuids":["F2WRtj2azjcbItsWO8"],"is_webmail":false}`
### What did you do?
Nothing special, just the default integration configuration
### What did you see?
Errors described in the first secction
### What did you expect to see?
no errors
### Anything else?
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.