Feature Request: Support for hooking different tokenizer or configuring existing for enhance text search capabilities

In the latest release, there is a way to use the text search index: https://docs.pinot.apache.org/basics/indexing/text-search-support#text-parsing-and-tokenization

However, currently, it provides only `Lucene's standard english text tokenizer` and configuration options for including/excluding of stop words.

There are certain domain-specific use cases where the above standard tokenizer won't suffice.
As an example,
- for the text `abc.pqr.xyz`, would like to split tokens using `.` along with existing `space` or `tab`. Here, the expectation is to get three tokens - `abc`, `pqr`,`xyz`
- for the text `GET /api/v1/customer`, would like split using `/`, and expect `GET`, `api`, `v1`, `customer`

However, currently, there is no way to include additional split chars for generating tokens in the existing tokenizer along with existing or to use another tokenizer.

As part of this ticket:
- Can we provide a way of extending the existing tokenizer?
- Can you also consider providing a way to configure a different tokenizer or hooking to a custom tokenizer?

Some discussion: https://apache-pinot.slack.com/archives/CDRCA57FC/p1677766557802739