elastic / elastic/integrations

[bug-hunter] zscaler_zia Email DLP rejects numeric timestamp format

Open Beginner friendly
#19,729 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
3d 4h
Merged PRs (30d)
209

Description

Package/path: `packages/zscaler_zia/data_stream/email_dlp/elasticsearch/ingest_pipeline/default.yml` lines 216-305.

Bug hypothesis: the new Email DLP pipeline attempts to support numeric timestamps with `yyyy-mm-dd HH:mm:ss`, but Java/Elasticsearch date patterns use `MM` for month and `mm` for minute. A valid value such as `2026-06-24 12:34:56` will not match any listed format, so each date processor enters `on_failure` and removes the corresponding timestamp field. For `zscaler_zia.email_dlp.time`, this also prevents `@timestamp` from being copied at lines 309-313.

Why user-impacting: users whose NSS Email DLP feed emits numeric date strings lose the event time and related email timestamps, producing ingest errors and incorrect/default event timestamps.

Minimal reproduction idea: add one pipeline test event with `time`, `feed_time`, `email.mail_sent_time`, `email.zs_rcv_time`, and `email.zs_sent_time` set to `2026-06-24 12:34:56` and `tz: GMT`; expect parsed ISO dates and `@timestamp`, but current pipeline removes the date fields and appends date-processor errors. Local formatter check reproduced the issue: `DateTimeFormatter.ofPattern("yyyy-mm-dd HH:mm:ss")` fails on `2026-06-24 12:34:56`, while `yyyy-MM-dd HH:mm:ss` parses successfully.

Evidence: the lowercase pattern appears at lines 224, 242, 260, 278, and 296, and each processor removes the field on failure at lines 227-305.

---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Bug Hunter](https://github.com/elastic/integrations/actions/runs/28096624959)

Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Jul 1, 2026, 12:11 PM UTC

Contributor guide

Open the contributing guide

Research direction

Start in packages/zscaler_zia/data_stream/email_dlp/elasticsearch/ingest_pipeline/default.yml, reviewing the date processors at lines 216-305 and the @timestamp copy at lines 309-313. Add the described pipeline test event with numeric timestamps and tz set to GMT, then verify the dates are parsed as ISO values, @timestamp is populated, and no date-processor errors or field removals occur.

Written by the indexing model from the issue text.

Assessment

Tech stack
elasticsearch
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.