elastic / elastic/integrations
[Cisco ASA]: Grok coverage census — 2 dead SGT processors, a shadowed pattern, and 8 message forms no pipeline test exercises
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 209
Description
Note: This issue has been drafted with :robot: Cursor/Claude Fable 5.1 under my supervision.
### Integration Name
Cisco ASA [cisco_asa]
### Dataset Name
cisco_asa.log
### Integration Version
cisco_asa 2.46.0
### What did you do?
Ran a grok coverage census over the package's pipeline tests: `cisco_asa` 2.46.0, `log` data stream, **65 grok processors, 109 patterns, 741 pipeline-test lines.** Every test line was run through the installed pipeline on a real Elasticsearch with `verbose: true`, recording for each grok step which pattern won and which captures fired on the value that step actually received (not the raw line — 64 of the 65 groks read a derived field). Every never-fired pattern was then checked for *reachability* in isolation, with Elasticsearch's own grok as the judge, and a sample line was crafted for every gap that turned out reachable and run end-to-end to confirm what the pipeline produces. Tooling notes are at the end.
### What did you see?
Three classes of unexercised pattern, which need different fixes:
| class | what it means | count | fix |
|---|---|---|---|
| **A. cannot be called** | no input exists that reaches it — dead code | 2 processors, 1 pattern | remove |
| **B. not called by the test data** | a real message form the package handles but no test line has | 2 processors, 2 patterns, 6 optional groups | add the lines below |
| **C. fallback only** | reachable only when the primary pattern fails | 1 pattern | decide whether the degradation is intended; test it |
## A. Cannot be called
### `grok_cisco_source_sgt` and `grok_cisco_destination_sgt` — no writer since they were added
```yaml
- grok:
field: "_temp_.cisco.source_sgt"
tag: "grok_cisco_source_sgt"
if: 'ctx?._temp_?.cisco?.source_sgt != null'
```
Nothing in the pipeline writes `_temp_.cisco.source_sgt` or `_temp_.cisco.destination_sgt` — no `%{…:_temp_.cisco.source_sgt}` capture, no `set`, no `rename`, and no data-driven path either: of the eleven painless scripts, the only writes into `_temp_` are `full_message`, `tz`, `cisco.dn_parts`, `cisco.message_id` and the literal key `cisco['security']` (the FTD mapping table's 94 targets contain no `sgt`), and the two `kv` processors target the sub-objects `_temp_.orig_security` and `_temp_.cisco.dn_parts`, which the `if` does not look at. The `@custom` pipeline runs after this one, so it cannot feed a processor in the middle of it. Both processors were introduced in #8205 (Oct 2023) in this state; `git log -S` finds no commit that ever wrote the field. The SGT work they look like they do is done elsewhere: `grok_cisco_source_user_or_sgt` p0 (`%{CISCO_DOMAIN_USER…}%{CISCO_SGT}`) wins on 21 test lines and fills `source_user_security_group_tag{,_name}`.
**Recommendation:** delete both processors and their `pattern_definitions`. If a separate SGT-only path was intended, it needs a writer and a test line — as it stands there is no behaviour to preserve.
### `grok_source_user_name` p2 `\*+` — shadowed by p1 on every input
```yaml
patterns:
- "(%{CISCO_DOMAIN})?%{CISCO_USER_EMAIL:source.user.email}"
- "(%{CISCO_DOMAIN})?%{CISCO_USER}"
- "\\*+"
pattern_definitions:
USERNAME: "[^@$]+"
CISCO_USER: "%{USERNAME:source.user.name}"
```
The grok processor picks the pattern that matches **earliest in the string**, and breaks ties by list order (elastic/elasticsearch#97283). `[^@$]+` matches any run of `*` at the same position `\*+` would, so p1 wins every tie. Checked by running the three patterns and their definitions through `_text_structure/test_grok_pattern` against `*****`, `***`, `*`, `@*****`: the second pattern wins every time. The intent was presumably ASA's `logging hide username` output (`user = *****`); it is handled — by p1, which writes `*****` back into `source.user.name`. p2 captures nothing, so deleting it changes no output.
**Recommendation:** delete p2. If the actual intent was to *drop* masked usernames rather than keep `*****`, that is a `remove` with `if: ctx.source?.user?.name ==~ /^\*+$/`, and a separate decision.
## B. Reachable, but no test line gets there
Each of these was confirmed by crafting a line and running it through the full pipeline; the fields shown are what came out.
### `parse_716002` p0 and `parse_716003` p0 — Cisco's documented form is the untested one
Both processors have two patterns: `Group <%{…}> User <%{…}> IP <%{IP}>` and the bare `Group %{NOTSPACE} User %{NOTSPACE} IP %{IP}`. Only the bare form appears in tests (2 lines each). The angle-bracket form is what ASA devices actually emit for the 716xxx family — the package's own tests say so: `parse_svc_message`'s bracketed pattern wins on 20 of 24 lines, `parse_716059`'s on 3 of 4. It is reachable (p1 fails at `IP <` because `<` is not an IP; p0 then matches) and parses completely:
```
%ASA-6-716002: Group User IP <10.1.1.5> WebVPN session terminated: User Requested.
→ parse_716002 p0; cisco.asa.webvpn.group_name=DfltGrpPolicy source.user.name=jdoe source.ip=10.1.1.5 event.reason="User Requested"
%ASA-6-716003: Group User IP <10.1.1.5> WebVPN access GRANTED: http://intranet.example.com/
→ parse_716003 p0; url.original/url.domain/url.path filled
```
### `parse_305011` `(idfw_user)` — documented, never present
Cisco's format string for 305011 is `Built {dynamic|static} {TCP|UDP|ICMP} translation from iface:real_addr/real_port [(idfw_user)] to iface:mapped_addr/mapped_port` — the optional identity-firewall user is in the pattern (`(\(%{NOTSPACE:source.user.name}\))?`), and none of the 58 test lines for 305011 carries one.
```
%ASA-6-305011: Built dynamic TCP translation from inside:10.0.0.5/49152(LOCAL\jdoe) to outside:203.0.113.1/49152
→ source.user.name=jdoe (domain stripped downstream by grok_source_user_name), source/destination complete
```
### `grok_client_address` and `grok_server_address` — reachable via 338301 and 43000x, neither tested
`client.address` / `server.address` are written by `set` for message 338301 (botnet DNS intercept) and by the FTD key/value script for `originalClientSrcIP` (430002/430003). There is no test line for any of those ids, so the two groks that split them into `.ip`/`.domain` have never run.
```
%ASA-6-338301: Intercepted DNS reply for domain www.example.com from outside:203.0.113.53/53 to inside:10.0.0.5/49152, matched Botnet-List
→ client.ip=10.0.0.5 server.ip=203.0.113.53 source.domain=www.example.com cisco.asa.rule_name=Botnet-List
```
Side finding from that line: `client.port` and `server.port` come out as **strings** (`"53"`, `"49152"`) — they are `set` from `{{{destination.port}}}` after the port `convert`s ran for source/destination but not for client/server. Worth a `convert … type: long`.
### `parse_302xxx` p6 — interface-prefixed `gaddr`, and a naming bug behind it
`gaddr (?:%{MAPPEDSRC}|%{NOTCOLON:_temp_.cisco.gaddr_interface}:%{MAPPEDSRC})`: the second alternative is reachable (the first fails on `outside:` because it then wants `/`), and no test line has an interface prefix on `gaddr`. The crafted line reaches it — and shows a bug:
```
%ASA-6-302021: Teardown ICMP connection for faddr outside:203.0.113.10/0 gaddr outside:198.51.100.1/0 laddr inside:10.0.0.5/0 type 8 code 0
→ cisco.asa.gaddr_interface=outside cisco.asa.source_interface=["outside","inside"]
```
`faddr`'s prefix and `laddr`'s prefix are both captured as `_temp_.cisco.source_interface`, so the field becomes an array. `faddr` is the foreign (destination) address — its prefix should be `_temp_.cisco.destination_interface`. This is the same faddr/laddr role confusion as #20839, one level up: whoever fixes the address mapping there should take the interface prefixes with it.
### Alternation branches that only ever went one way
- `parse_106010`: `((protocol %{POSINT:network.iana_number})|%{NOTSPACE:network.transport})` — p0 has only ever seen the `transport` branch, p1 only the `protocol` branch; p1's `source.port`/`destination.port` never captured. 2 test lines total for 106010. (#16950 is the same `protocol N` vs transport-name alternation biting on 106023; that one *is* covered here — both branches fire across its 79 lines — which is what a fix with a test looks like.)
- `parse_313005` p1: `source.port`, `destination.port` never captured.
- `parse_302013-302015_outbound`: the `(\(%{CISCO_USER_OR_SGT_SRC}\))?` / `_DST` groups and `( \(%{CISCO_USER:_temp_.cisco.termination_user}\))?` never fire on an outbound line. All three are real forms — the **inbound** twin of this processor fires them on eleven existing test lines (`test-sample.log`, `test-asa-fix.log`, `test-sgt-tag-name.log`: `… to inside:89.160.20.112/9803 (89.160.20.112/9803) (bob)`), and the same construct is in cisco_ftd. The outbound direction simply has no sample with a user on it. Crafted `… (10.0.0.5/49152) (jdoe)` → `cisco.asa.termination_user=jdoe`; it is the eighth line below.
## C. Reachable only as a fallback
### `parse_751025` p1 — catches the username when p0 rejects the line
p0 is anchored: `^Local:… Remote:… Username:%{NOTSPACE}(?: Group:…)?(?: IPv4 Address=…)?(?: IPv6 address=…)?(?: assigned to session)?\s*$`. Any trailing token outside that list makes p0 fail and hands the line to p1 (`^%{DATA} Username:%{NOTSPACE:source.user.name}\s*%{GREEDYDATA}$`), which keeps the username and **loses Local/Remote addresses and ports**:
```
%ASA-6-751025: Local:10.0.0.1:500 Remote:10.0.0.2:500 Username:jdoe IKEv2 Group:VPN-Users IPv4 Address=10.10.10.5 assigned to session, Tunnel-Group:VPN-Users
→ parse_751025 p1; document has source.user.name=jdoe and nothing else from the message
```
**Recommendation:** either make p0's tail tolerant (`(?: assigned to session)?%{GREEDYDATA}` instead of `\s*$`) so a new suffix costs nothing, or keep the fallback and add a test line that documents the degradation. Today the fallback exists and nothing shows what it drops.
## Proposed test data
Eight lines, one per gap, each verified end-to-end above. Suggested as `_dev/test/pipeline/test-coverage-gaps.log`, expected output generated with `elastic-package test pipeline -g`:
```
Jan 23 15:10:55 10.10.10.1 %ASA-6-716002: Group User IP <10.1.1.5> WebVPN session terminated: User Requested.
Jan 23 15:10:55 10.10.10.1 %ASA-6-716003: Group User IP <10.1.1.5> WebVPN access GRANTED: http://intranet.example.com/
Jan 23 15:10:55 10.10.10.1 %ASA-6-751025: Local:10.0.0.1:500 Remote:10.0.0.2:500 Username:jdoe IKEv2 Group:VPN-Users IPv4 Address=10.10.10.5 assigned to session, Tunnel-Group:VPN-Users
Jan 23 15:10:55 10.10.10.1 %ASA-6-113005: AAA user authentication Rejected : reason = Invalid password : server = 10.0.0.3 : user = ***** : user IP = 10.1.1.5
Jan 23 15:10:55 10.10.10.1 %ASA-6-305011: Built dynamic TCP translation from inside:10.0.0.5/49152(LOCAL\jdoe) to outside:203.0.113.1/49152
Jan 23 15:10:55 10.10.10.1 %ASA-6-302013: Built outbound TCP connection 12345 for outside:203.0.113.10/443 (203.0.113.10/443) to inside:10.0.0.5/49152 (10.0.0.5/49152) (jdoe)
Jan 23 15:10:55 10.10.10.1 %ASA-6-302021: Teardown ICMP connection for faddr outside:203.0.113.10/0 gaddr outside:198.51.100.1/0 laddr inside:10.0.0.5/0 type 8 code 0
Jan 23 15:10:55 10.10.10.1 %ASA-6-338301: Intercepted DNS reply for domain www.example.com from outside:203.0.113.53/53 to inside:10.0.0.5/49152, matched Botnet-List
```
The 113005 line with `*****` is included not for the dead `\*+` pattern but to pin the behaviour p1 provides for masked usernames, so removing p2 has a test behind it.
### What did you expect to see?
Recommended changes:
1. Remove `grok_cisco_source_sgt`, `grok_cisco_destination_sgt` (dead since #8205).
2. Remove `\*+` from `grok_source_user_name` (unreachable; no output change).
3. Fix `parse_302xxx` p6: `faddr`'s interface prefix → `_temp_.cisco.destination_interface`, not `source_interface` (currently produces an array).
4. `convert` `client.port` / `server.port` to `long` for 338301, matching source/destination.
5. Loosen `parse_751025` p0's tail, or test the p1 degradation.
6. Add the eight test lines above.
## On branches: what this census can and cannot see
The question came up whether alternations *inside* a pattern — `(a|b)` — are reviewed for completeness. Partly. Coverage here is capture-based: a branch is known to have run if a capture inside it fired. Across the 61 executed processors' winning patterns and their custom definitions, cisco_asa has **239 alternatives: 85 with a capture that fired, 13 with captures that never fired (all listed above), and 141 — 59% — with no capture at all**, which this method cannot see. Those are things like `(?:for|from)`, `(?:state-bypass )?`, `resumed connection|resumed. Connection` in 716059, and the `(?:%{FADDR}|%{NOTCOLON}:%{FADDR})` alternatives in 302020 where the interface prefix is deliberately not captured.
They are not unreachable in principle: wrapping each alternative in a uniquely named group and reading Elasticsearch's own offsets back (which is how the exact-span measurement already works for tokens) would make branch coverage exact without reimplementing any regex. Two follow-ups on the tooling side, tracked separately: that branch instrumentation, and a mechanical shadow check — generate inputs *from* each pattern and see whether it ever wins against the ones before it — so class A is found by machine across all 109 patterns rather than by reading. If that turns up more than `\*+`, I will add it here.
## How this was done
Two pieces of Integration Experience tooling (internal for now), driven from a short script against a local Elasticsearch:
- **Log Shape** simulates a line through the *installed* package pipeline with `verbose: true` and traces every extracted value back to its exact characters in the line. To measure grok steps exactly it sends each one to `_text_structure/test_grok_pattern` with the value the step actually received — `_temp_.full_message` after the syslog envelope is stripped, `source.user.name` after 305011 fills it. The census taps that hook: per executed grok step, the input, the winning pattern and the captures that fired. Testing patterns against raw lines would get cisco_asa entirely wrong; 45 of its 65 groks read `message`, which is itself what `grok_full_message` cut out of the envelope. The same run yields the final document, which is how the `source_interface` array and the string ports surfaced without looking for them.
- **Grok Visualizer** answers the isolation questions — given these three patterns and their definitions, which one wins on `*****`? — by instrumenting the pattern so Elasticsearch reports every token's exact range, and by bisecting a non-matching pattern down to the token that failed. That is what turned "never wins in the test data" into "cannot win" in four calls.
One note for whoever touches `grok_message_repeats`: its hand-written `(?\d+)` is valid — Elasticsearch splits every group name on `:`, so it behaves exactly like `%{INT:_temp_.cisco.message_repeats:int}` — and it works on the one test line that reaches it. It did confuse the tooling until this census, which is a reminder that a census over every package's test lines tests the tools as much as the packages.
Contributor guide
Research direction
Start with the Cisco ASA pipeline processors named in the issue, especially grok_cisco_source_sgt, grok_cisco_destination_sgt, grok_source_user_name, parse_302xxx, and parse_751025. Add the proposed lines to _dev/test/pipeline/test-coverage-gaps.log and run elastic-package test pipeline -g; done means the dead patterns are removed, the reachable forms are exercised, and the documented interface and port mappings are corrected or explicitly covered.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- elasticsearch
- Domain
- backend, testing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100