Six malformed `%{…}` in shipped grok patterns are read as literal text, so five fields are never captured

## Summary

Six `%{…}` in shipped grok patterns are not tokens by Grok.java's grammar, so Elasticsearch reads them as literal regex text. Five of them mean a field the pattern advertises is **never captured**, and nothing anywhere reports a problem. The sixth is harmless but wrong.

All six are on `main` (`1be4c5d`), and each is verified against Elasticsearch 9.4.4 with `_ingest/pipeline/_simulate` rather than argued from the grammar.

Note: drafted with :robot: Cursor/Opus 5, under my supervision.

| # | package | file | line |
|---|---|---|---|
| 1 | `citrix_adc` 1.19.1 | `data_stream/log/…/sslvpn_and_aaatm_feature.yml` | 167 |
| 2 | `hid_bravura_monitor` 1.21.1 | `data_stream/log/…/default.yml` | 85 |
| 3 | `nats` 1.12.0 | `data_stream/log/…/default.yml` | 41 |
| 4 | `stan` 1.11.0 | `data_stream/log/…/default.yml` | 41 |
| 5 | `pulse_connect_secure` 2.6.1 | `data_stream/log/…/default.yml` | 62 |
| 6 | `iptables` 1.23.1 | `data_stream/log/…/default.yml` | 73 |

## Why none of these fail loudly

Grok treats `%{…}` as a token only when it matches [`GROK_PATTERN` in `Grok.java`](https://github.com/elastic/elasticsearch/blob/main/libs/grok/src/main/java/org/elasticsearch/grok/Grok.java#L35), whose pattern name and field name are narrow:

```java
+ "(?[A-z0-9]+)"
+ "(?::(?[[:alnum:]@\\[\\]_:.-]+))?"
```

Text that does not match is left in the regex source untouched, `%{` and all. So a typo *inside* a `%{…}` never produces `Unable to find pattern [X]`: the processor compiles and runs, and either the pattern stops matching (the log would have to contain `%{…}` literally) or it matches with the capture absent.

Note the asymmetry, which is what makes this a trap rather than a typo: `%{WORD_tmp.outcome}` is silent **because** of the `.`. Had it been `%{WORD_tmp}`, the name would have parsed, the lookup would have failed, and the pipeline would have refused to install.

Each of the five also has something nearby that absorbs the miss — a `%{DATA}` catch-all as the last pattern, or `ignore_failure: true` — so there is no `_grokparsefailure` either.

---

## 1. `citrix_adc` — `|` inside a pattern name

`packages/citrix_adc/data_stream/log/elasticsearch/ingest_pipeline/sslvpn_and_aaatm_feature.yml:167`

```yaml
- '^Logout handler : %{DATA}, for user <%{USERNAME|EMAILADDRESS:citrix_adc.log.username}>$'
- '^aaatm_handler successfully parsed assertion client ip is %{IP:citrix_adx.log.client_ip}, username is %{DATA:citrix_adc.log.user}$'
- '%{DATA}'
```

`|` cannot appear in a pattern name, so `%{USERNAME` stays literal and that `|` becomes an alternation of the **whole pattern**:

```
^Logout handler : (?:.*?), for user <%{USERNAME | EMAILADDRESS:citrix_adc.log.username}>$
```

`Logout handler : session closed, for user ` does not match. The only lines that do are ones carrying the grok source text, and they capture nothing:

```
MATCHED: Logout handler : x, for user
MATCHED: Logout handler : session closed, for user <%{USERNAME
```

The third pattern, `%{DATA}`, then matches everything, so the processor never fails.

This one has a paper trail in the package's own tests. `_dev/test/pipeline/test-citrix-native.json` contains exactly the line this rule is for:

```
… SSLVPN Message 600000 0 : Logout handler : starting 30sec timer after sending saml logout req to IdP, for user
```

and the committed `test-citrix-native.json-expected.json` for that document has `citrix_adc.log: {"message": …}` and **no `username`**. The test is green because the expectation was generated from the broken output — so the fix will need that file regenerated.

**Fix** (verified: captures `user_name@domain.com`, and plain `user_name` too):

```yaml
- '^Logout handler : %{DATA}, for user <(?:%{EMAILADDRESS:citrix_adc.log.username}|%{USERNAME:citrix_adc.log.username})>$'
```

While in that processor: line 168 says `citrix_adx.log.client_ip`, which looks like a typo for `citrix_adc` — it is the only `citrix_adx` in the package, and the field is not declared in `fields.yml`.

## 2. `hid_bravura_monitor` — a character class where a pattern name goes

`packages/hid_bravura_monitor/data_stream/log/elasticsearch/ingest_pipeline/default.yml:85`

```yaml
- grok:
field: pslogid
patterns:
- '%{UUID:hid_bravura_monitor.request.id}'
- '%{[A-Fa-f0-9]{32}:hid_bravura_monitor.request.id}'
ignore_missing: true
ignore_failure: true
description: Set requestid if batchid
```

`-` is not allowed in a pattern name, so the second pattern is regex: literal `%{`, then 32 hex characters, then literal `:hid_bravura_monitor` and so on. Verified:

```
0123456789abcdef0123456789abcdef → no match
%{0123456789abcdef0123456789abcdef:hid_bravura_monitor.request.id} → matches, captures nothing
```

So the batchid branch never fires; `ignore_failure: true` hides it and the `%{UUID}` branch above still works, which is presumably why it went unnoticed.

**Fix** — a raw named group, since the intent is an inline regex:

```yaml
- '(?[A-Fa-f0-9]{32})'
```

## 3 and 4. `nats` and `stan` — a missing `}`

`packages/nats/data_stream/log/elasticsearch/ingest_pipeline/default.yml:41` and `packages/stan/…/default.yml:41` (same pattern, same line, the two packages share a lineage):

```yaml
- '%{NATSDIRECTION:network.direction} \[%{NATSERROR:nats.log.msg.type}\s+%{GREEDYDATA:nats.log.msg.error\]'
```

`\` is not allowed in a field name, so there is no closing `}` and the token ends up as literal text. The pattern therefore never matches an error line at all:

```
<<- [-ERROR 'authorization violation'

as shipped no match
with the `}` network.direction '<<-', nats.log.msg.type '-ERROR',
nats.log.msg.error "'authorization violation'"
```

**Fix** — move the brace:

```yaml
- '%{NATSDIRECTION:network.direction} \[%{NATSERROR:nats.log.msg.type}\s+%{GREEDYDATA:nats.log.msg.error}\]'
```

## 5. `pulse_connect_secure` — a missing `:`

`packages/pulse_connect_secure/data_stream/log/elasticsearch/ingest_pipeline/default.yml:62`

```yaml
- 'Login %{WORD:_tmp.outcome}( %{GREEDYDATA})?. Reason: %{GREEDYDATA:event.reason}'
- '^Primary authentication %{WORD_tmp.outcome}'
```

The line above it shows the intent. As written the pattern matches only text that literally contains `%{WORD_tmp.outcome}`, and captures nothing when it does:

```
'Primary authentication %{WORD_tmp.outcome}' → matches, no fields
'Primary authentication failed' → no match
```

`ignore_failure: true` on the processor, so `Primary authentication …` lines simply get no outcome.

**Fix**:

```yaml
- '^Primary authentication %{WORD:_tmp.outcome}'
```

## 6. `iptables` — wrong, but harmless

`packages/iptables/data_stream/log/elasticsearch/ingest_pipeline/default.yml:73`

```yaml
ECS_SYSLOG_PRI: '<%{NONNEGINT:log.syslog.priority>'
```

`>` where `}` belongs. The processor 20 lines above spells the same definition correctly (`'<%{NONNEGINT:log.syslog.priority:long}>'`), and none of this processor's five patterns reference `ECS_SYSLOG_PRI` — so it is dead weight rather than a bug. Worth fixing or deleting so it is not copied onward.

---

## How these were found, and a suggested lint

Scan the `patterns` and `pattern_definitions` of every grok processor, and at each `%{` check whether Grok's own token grammar matches there:

```python
TOKEN = re.compile( # transcribed from Grok.java's GROK_PATTERN;
r"%\{" # [A-z0-9] is the ASCII range, so it admits
r"(?P" # [ \ ] ^ _ and ` as well as letters and digits
r"(?P[A-Za-z0-9_\[\\\]^`]+)"
r"(?::(?P[A-Za-z0-9@\[\]_:.\-]+))?"
r")"
r"(?:=(?P(?:[^{}]+|\.+)+))?"
r"\}"
)

for m in re.finditer(r"%\{", text):
if not TOKEN.match(text, m.start()):
report(where, text[m.start():m.start() + 60])
```

Six hits over the whole repo, no false positives — the signal is clean enough for `elastic-package` to reject it at build time, and it is the only check that would have caught any of these. A capture that silently never happens is invisible to pipeline tests too, since the expected documents are generated from actual output.

I am happy to send the fixes, one PR per package, if that is the preferred shape — say the word and whether the `citrix_adc` expected-document regeneration should come with it.

_{Found while building a tool that reports exact character offsets for grok matches; the census that turned these up covers 2917 patterns across this repo.}

Start with Grok.java's GROK_PATTERN, then inspect the six listed pipeline files in the citrix_adc, hid_bravura_monitor, nats, stan, pulse_connect_secure, and iptables packages. Apply the documented corrections, regenerate _dev/test/pipeline/test-citrix-native.json-expected.json for citrix_adc, and run the affected package pipeline tests. Done means the intended fields are captured and the malformed tokens no longer remain.

Written by the indexing model from the issue text.