GROK: inconsistencies between in-yaml and pattern_paths

Hi there,
I spent some time debugging a grok issue which worked perfectly when all patterns were defined in the yaml, but failed when the patterns were included via pattern_paths. root cause AFAICT is how underscores are handled differently.
they are allowed in the yaml, but not in the included patterns.

here's my test case:
csv input file

id,log
1,GROK 1 test
2,GROK 2 test
3,KROG 3 test
4,KROG 4 test

yaml without included patterns

pipeline:
  threads: 1
  processors:
    - label: "branch_test"
      branch:
        request_map: |
          root = this.log
        processors:
          - label: "grok_test"
            grok:
              expressions:
                - '%{GROK_1}'
                - '%{GROK1}'
              # pattern_paths:
                 # - "./grok"
              pattern_definitions: 
                PRE_1: '^GROK'
                PRE1: '^KROG'
                GROK_1: '%{PRE_1} %{INT:int} %{GREEDYDATA:test}'
                GROK1: '%{PRE1} %{INT:int} %{GREEDYDATA:test}'                       
        result_map: |
          root.branchresult = this

results in:

{"branchresult":{"int":"1","test":"test"},"id":"1","log":"GROK 1 test"}
{"branchresult":{"int":"2","test":"test"},"id":"2","log":"GROK 2 test"}
{"branchresult":{"int":"3","test":"test"},"id":"3","log":"KROG 3 test"}
{"branchresult":{"int":"4","test":"test"},"id":"4","log":"KROG 4 test"}

Nice!

but moving these patterns out to a patterns.grok file - and (un)commenting the relevant bits in the yaml

# TESTPATS
PRE_1 ^GROK
PRE1  ^KROG

returns:

{"branchresult":{"int":"1","test":"test"},"id":"1","log":"GROK 1 test"}
{"branchresult":{"int":"2","test":"test"},"id":"2","log":"GROK 2 test"}
ERRO Branch error: processors failed: no pattern matches found  @service=benthos label=branch_test path=root.pipeline.processors.0
{"id":"3","log":"KROG 3 test"}
ERRO Branch error: processors failed: no pattern matches found  @service=benthos label=branch_test path=root.pipeline.processors.0
{"id":"4","log":"KROG 4 test"}

I didn't know that _ isn't (usually) allowed in grok so it took me a fair bit of time to pin down.
It's not a bug perse, so just reporting it here for you to decide on whether to (dis)allow _, mention it in the docs, and/or catch it in the linter.

On that note, there's no linting of the included patterns so a bit of gibberish in there causes a hard crash.
(And while I am at it, another one that tripped me up was a trailing space in the PRE pattern in the included file ;-(

finally, there's something off/unexpected with the globbing in pattern_paths.
my file is named ./grok/patterns.grok
"./grok" works , see earlier
"./grok/*.grok" does not work
I know it says pattern_paths but still...

My apologies for dumping all of this in a single issue.
Once I got the underscore and space sorted it was actually smooth sailing all the way. Thanks!

Start with the grok processor's handling of pattern_definitions and pattern_paths, then reproduce the supplied YAML and patterns.grok examples. Check the underscore, malformed-pattern, trailing-space, and globbing cases; done requires a decided and tested behavior, with documentation or linting changes if those are selected.