elastic / elastic/integrations
[bug-hunter] zscaler_zia Email DLP rejects numeric timestamp format
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 209
Description
Package/path: `packages/zscaler_zia/data_stream/email_dlp/elasticsearch/ingest_pipeline/default.yml` lines 216-305.
Bug hypothesis: the new Email DLP pipeline attempts to support numeric timestamps with `yyyy-mm-dd HH:mm:ss`, but Java/Elasticsearch date patterns use `MM` for month and `mm` for minute. A valid value such as `2026-06-24 12:34:56` will not match any listed format, so each date processor enters `on_failure` and removes the corresponding timestamp field. For `zscaler_zia.email_dlp.time`, this also prevents `@timestamp` from being copied at lines 309-313.
Why user-impacting: users whose NSS Email DLP feed emits numeric date strings lose the event time and related email timestamps, producing ingest errors and incorrect/default event timestamps.
Minimal reproduction idea: add one pipeline test event with `time`, `feed_time`, `email.mail_sent_time`, `email.zs_rcv_time`, and `email.zs_sent_time` set to `2026-06-24 12:34:56` and `tz: GMT`; expect parsed ISO dates and `@timestamp`, but current pipeline removes the date fields and appends date-processor errors. Local formatter check reproduced the issue: `DateTimeFormatter.ofPattern("yyyy-mm-dd HH:mm:ss")` fails on `2026-06-24 12:34:56`, while `yyyy-MM-dd HH:mm:ss` parses successfully.
Evidence: the lowercase pattern appears at lines 224, 242, 260, 278, and 296, and each processor removes the field on failure at lines 227-305.
---
[What is this?](https://ela.st/github-ai-tools) | [From workflow: Bug Hunter](https://github.com/elastic/integrations/actions/runs/28096624959)
Give us feedback! React with 🚀 if perfect, 👍 if helpful, 👎 if not.
> - [x] expires on Jul 1, 2026, 12:11 PM UTC
Contributor guide
Research direction
Start in packages/zscaler_zia/data_stream/email_dlp/elasticsearch/ingest_pipeline/default.yml, reviewing the date processors at lines 216-305 and the @timestamp copy at lines 309-313. Add the described pipeline test event with numeric timestamps and tz set to GMT, then verify the dates are parsed as ISO values, @timestamp is populated, and no date-processor errors or field removals occur.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 84/100