elastic / elastic/integrations
[checkpoint] Date processor fails to parse CME `creation_time` timestamp format (`yyyy-MM-dd HH:mm:ss.SSSSSS`)
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 209
Description
### Integration Name
Check Point [packages/checkpoint]
### Dataset Name
_No response_
### Integration Version
1.49.1
### Agent Version
9.4.2
### Agent Output Type
elasticsearch
### Elasticsearch Version
9.4.2
### OS Version and Architecture
Unknown
### Software/API Version
Check Point Cloud Management Extension (CME)
### Error Message
```
Processor 'conditional' with tag 'date_checkpoint_creation_time_to_event_start_f2bceb79'
in pipeline 'logs-checkpoint.firewall-1.49.1' failed with message 'Could not fully parse datetime'
```
### Event Original
```
<134>1 2026-08-21T14:15:20Z firewall-host CheckPoint 96397 - [alert:"cme_alerting"; flags:"139328"; ifdir:"inbound"; loguid:"{0x6a885d78,0xfd,0x96fcfe0a,0x3c56f88d}"; origin:"192.0.2.1"; sequencenum:"1"; time:"1787321720"; version:"5"; calc_desc:"Error during synchronization with Security Gateways."; creation_time:"2026-08-21 10:14:59.626375"; product:"CME"; severity:"Critical"]
```
### What did you do?
Deployed the Check Point integration (v1.49.1) using Elastic Agent (v9.4.2) to collect firewall logs via UDP syslog. Check Point Cloud Management Extension (CME) events are included in the log stream alongside standard firewall events.
### What did you see?
CME events are stored with `event.kind: "pipeline_error"` and `event.start` is never populated. The `error.message` field on the affected documents reads:
```
Processor 'conditional' with tag 'date_checkpoint_creation_time_to_event_start_f2bceb79' in pipeline 'logs-checkpoint.firewall-1.49.1' failed with message 'Could not fully parse datetime'
```
The affected field is `checkpoint.creation_time`, which the CME subsystem emits in the format `yyyy-MM-dd HH:mm:ss.SSSSSS` — for example:
```
creation_time: "2026-08-21 10:14:59.626375"
```
Inspecting the pipeline confirms the `date` processor responsible for parsing this field:
```json
{
"date": {
"tag": "date_checkpoint_creation_time_to_event_start_f2bceb79",
"formats": ["ISO8601", "UNIX"],
"field": "checkpoint.creation_time",
"target_field": "event.start",
"if": "ctx.checkpoint?.creation_time != null"
}
}
```
The CME-generated value has three characteristics that prevent it from matching either accepted format:
- Uses a **space** instead of `T` as the date/time separator
- Has **no timezone offset** or `Z` suffix
- Uses **6-digit (microsecond) fractional seconds**
Because there is no `on_failure` handler on this processor, the parse failure propagates and the entire document is flagged as a pipeline error. The separate `checkpoint.time` epoch field (handled by the `checkpoint_time_conversion_script` processor earlier in the pipeline) is unaffected.
### What did you expect to see?
CME events should be ingested successfully with `event.start` populated from `checkpoint.creation_time`. The date processor should support the space-separated, microsecond-precision format that CME emits.
The expected fix is to expand the `formats` list and add an `on_failure` handler to prevent hard failures for any unrecognised future variants:
```yaml
- date:
tag: date_checkpoint_creation_time_to_event_start_f2bceb79
field: checkpoint.creation_time
formats:
- "ISO8601"
- "UNIX"
- "yyyy-MM-dd HH:mm:ss.SSSSSS"
- "yyyy-MM-dd HH:mm:ss.SSS"
- "yyyy-MM-dd HH:mm:ss"
target_field: event.start
if: "ctx.checkpoint?.creation_time != null"
on_failure:
- append:
field: error.message
value: "date-parse-failure-{{{ _ingest.on_failure_processor_tag }}}: {{{ _ingest.on_failure_message }}}"
```
The three explicit formats handle:
- `yyyy-MM-dd HH:mm:ss.SSSSSS` — CME microsecond precision (the failing case)
- `yyyy-MM-dd HH:mm:ss.SSS` — millisecond precision variant
- `yyyy-MM-dd HH:mm:ss` — whole-second variant
Existing ISO8601 and UNIX formats are preserved, so no regression for currently-working log types.
### Anything else?
- The `checkpoint.subs_exp` date processor in the same pipeline already accepts space-separated date formats (`"EEE MMM dd HH:mm:ss yyyy"`), confirming that non-ISO8601 date patterns are handled elsewhere in this integration. This is consistent with the same gap being addressed in the vSphere integration for an identical symptom ([elastic/integrations#](https://github.com/elastic/integrations)).
- Note for the maintainer: PR #19996 is currently open against this integration and appears to be narrowing `checkpoint.creation_time` formats to UNIX-only (flagged as unresolved in the automated review). If that PR merges in its current form, it would make this bug worse by also removing ISO8601 support. It may be worth coordinating the fix for this issue with that PR.
- A `@custom` pipeline cannot be used as a workaround here. Fleet custom pipelines execute only when the package pipeline succeeds; when the package pipeline errors, the `@custom` pipeline is skipped entirely.
Contributor guide
Research direction
Start at the Check Point firewall ingest pipeline under packages/checkpoint and inspect the date processor tagged date_checkpoint_creation_time_to_event_start_f2bceb79. Reproduce it with the provided CME event, then verify that supported space-separated timestamp variants populate event.start without producing a pipeline_error; coordinate with PR #19996 before changing overlapping formats.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch
- Domain
- observability-sre
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 68/100