elastic / elastic/integrations

[checkpoint] Date processor fails to parse CME `creation_time` timestamp format (`yyyy-MM-dd HH:mm:ss.SSSSSS`)

Open Beginner friendly
#20,925 1 comment 0 reactions 0 assignees View on GitHub
needs:triage
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
3d 4h
Merged PRs (30d)
209

Description

### Integration Name

Check Point [packages/checkpoint]

### Dataset Name

_No response_

### Integration Version

1.49.1

### Agent Version

9.4.2

### Agent Output Type

elasticsearch

### Elasticsearch Version

9.4.2

### OS Version and Architecture

Unknown

### Software/API Version

Check Point Cloud Management Extension (CME)

### Error Message

```
Processor 'conditional' with tag 'date_checkpoint_creation_time_to_event_start_f2bceb79'
in pipeline 'logs-checkpoint.firewall-1.49.1' failed with message 'Could not fully parse datetime'
```

### Event Original

```
<134>1 2026-08-21T14:15:20Z firewall-host CheckPoint 96397 - [alert:"cme_alerting"; flags:"139328"; ifdir:"inbound"; loguid:"{0x6a885d78,0xfd,0x96fcfe0a,0x3c56f88d}"; origin:"192.0.2.1"; sequencenum:"1"; time:"1787321720"; version:"5"; calc_desc:"Error during synchronization with Security Gateways."; creation_time:"2026-08-21 10:14:59.626375"; product:"CME"; severity:"Critical"]
```

### What did you do?

Deployed the Check Point integration (v1.49.1) using Elastic Agent (v9.4.2) to collect firewall logs via UDP syslog. Check Point Cloud Management Extension (CME) events are included in the log stream alongside standard firewall events.

### What did you see?

CME events are stored with `event.kind: "pipeline_error"` and `event.start` is never populated. The `error.message` field on the affected documents reads:

```
Processor 'conditional' with tag 'date_checkpoint_creation_time_to_event_start_f2bceb79' in pipeline 'logs-checkpoint.firewall-1.49.1' failed with message 'Could not fully parse datetime'
```

The affected field is `checkpoint.creation_time`, which the CME subsystem emits in the format `yyyy-MM-dd HH:mm:ss.SSSSSS` — for example:

```
creation_time: "2026-08-21 10:14:59.626375"
```

Inspecting the pipeline confirms the `date` processor responsible for parsing this field:

```json
{
"date": {
"tag": "date_checkpoint_creation_time_to_event_start_f2bceb79",
"formats": ["ISO8601", "UNIX"],
"field": "checkpoint.creation_time",
"target_field": "event.start",
"if": "ctx.checkpoint?.creation_time != null"
}
}
```

The CME-generated value has three characteristics that prevent it from matching either accepted format:

- Uses a **space** instead of `T` as the date/time separator
- Has **no timezone offset** or `Z` suffix
- Uses **6-digit (microsecond) fractional seconds**

Because there is no `on_failure` handler on this processor, the parse failure propagates and the entire document is flagged as a pipeline error. The separate `checkpoint.time` epoch field (handled by the `checkpoint_time_conversion_script` processor earlier in the pipeline) is unaffected.

### What did you expect to see?

CME events should be ingested successfully with `event.start` populated from `checkpoint.creation_time`. The date processor should support the space-separated, microsecond-precision format that CME emits.

The expected fix is to expand the `formats` list and add an `on_failure` handler to prevent hard failures for any unrecognised future variants:

```yaml
- date:
tag: date_checkpoint_creation_time_to_event_start_f2bceb79
field: checkpoint.creation_time
formats:
- "ISO8601"
- "UNIX"
- "yyyy-MM-dd HH:mm:ss.SSSSSS"
- "yyyy-MM-dd HH:mm:ss.SSS"
- "yyyy-MM-dd HH:mm:ss"
target_field: event.start
if: "ctx.checkpoint?.creation_time != null"
on_failure:
- append:
field: error.message
value: "date-parse-failure-{{{ _ingest.on_failure_processor_tag }}}: {{{ _ingest.on_failure_message }}}"
```

The three explicit formats handle:
- `yyyy-MM-dd HH:mm:ss.SSSSSS` — CME microsecond precision (the failing case)
- `yyyy-MM-dd HH:mm:ss.SSS` — millisecond precision variant
- `yyyy-MM-dd HH:mm:ss` — whole-second variant

Existing ISO8601 and UNIX formats are preserved, so no regression for currently-working log types.

### Anything else?

- The `checkpoint.subs_exp` date processor in the same pipeline already accepts space-separated date formats (`"EEE MMM dd HH:mm:ss yyyy"`), confirming that non-ISO8601 date patterns are handled elsewhere in this integration. This is consistent with the same gap being addressed in the vSphere integration for an identical symptom ([elastic/integrations#](https://github.com/elastic/integrations)).

- Note for the maintainer: PR #19996 is currently open against this integration and appears to be narrowing `checkpoint.creation_time` formats to UNIX-only (flagged as unresolved in the automated review). If that PR merges in its current form, it would make this bug worse by also removing ISO8601 support. It may be worth coordinating the fix for this issue with that PR.

- A `@custom` pipeline cannot be used as a workaround here. Fleet custom pipelines execute only when the package pipeline succeeds; when the package pipeline errors, the `@custom` pipeline is skipped entirely.

Contributor guide

Open the contributing guide

Research direction

Start at the Check Point firewall ingest pipeline under packages/checkpoint and inspect the date processor tagged date_checkpoint_creation_time_to_event_start_f2bceb79. Reproduce it with the provided CME event, then verify that supported space-separated timestamp variants populate event.start without producing a pipeline_error; coordinate with PR #19996 before changing overlapping formats.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch
Domain
observability-sre
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.