Proposal for Optimizing transformers.BertTokenizerFast

### Feature request

The `transformers.BertTokenizerFast` tokenizer, implemented in Rust, is excellent but has been shown to lag behind other implementations in performance.

For instance, [PaddleNLP's BertTokenizerFast](https://paddlenlp.readthedocs.io/en/stable/_modules/paddlenlp/experimental/faster_tokenizer.html), a `C++` implementation, provides identical results but performs slightly faster. Additionally, several other recent implementations show promising performance improvements.

For example, [FastBertTokenizer](https://github.com/georg-jung/FastBertTokenizer), developed in `C#`, demonstrates exceptional speed, although it does not offer a Python interface.

[BertTokenizerFlash](https://github.com/NLPOptimize/flash-tokenizer) is another significant `C++` implementation that provides a [Python package](https://pypi.org/project/flash-tokenizer/). According to performance benchmarks provided below, [BertTokenizerFlash](https://github.com/NLPOptimize/flash-tokenizer) is more than ten times faster than `transformers.BertTokenizerFast`. The baseline used for accuracy comparisons is Google's original [BertTokenizer](https://github.com/google-research/bert/blob/master/tokenization.py), with accuracy measured by the exact match rate of `input_ids`.

[BertTokenizerFlash](https://github.com/NLPOptimize/flash-tokenizer) achieves its high performance by utilizing the LinMaxMatching method based on the Aho–Corasick algorithm, as detailed in [Fast WordPiece Tokenization](https://arxiv.org/abs/2012.15524).

It would be beneficial if **HuggingFace** considered enhancing their current tokenizer implementation for improved speed and accuracy.

All the results below were measured using Python in a single-threaded execution environment.

#### bert-base-multilingual-cased
| Tokenizer | Elapsed Time | Texts | Accuracy |
|--------------------------------|--------------|-----------|-----------|
| BertTokenizerFast(Huggingface) | 215.7353s | 2,000,001 | 99.7964% |
| BertTokenizerFast(PaddleNLP) | 201.8254s | 2,000,001 | 99.7964% |
| FastBertTokenizer(Tensorflow) | 439.2282s | 2,000,001 | 99.7892% |
| Blingfire | 37.4688s | 2,000,001 | 99.9780% |
| **FlashBertTokenizer** | 21.9977s | 2,000,001 | 99.8971% |

#### bert-base-chinese
| Tokenizer | Elapsed Time | Texts | Accuracy |
|--------------------------------|--------------|-----------|-----------|
| BertTokenizerFast(Huggingface) | 148.0193s | 1,000,000 | 99.1475% |
| BertTokenizerFast(PaddleNLP) | 143.9290s | 1,000,000 | 99.1475% |
| Blingfire | 17.4364s | 1,000,000 | 61.1254% |
| **FlashBertTokenizer** | 10.8936s | 1,000,000 | 99.5057% |

#### kcbert-base
| Tokenizer | Elapsed Time | Texts | Accuracy |
|--------------------------------|--------------|-----------|-----------|
| BertTokenizerFast(Huggingface) | 52.6710s | 1,000,001 | 99.6754% |
| BertTokenizerFast(PaddleNLP) | 46.8151s | 1,000,001 | 99.6754% |
| FastBertTokenizer(Tensorflow) | 204.8551s | 1,000,001 | 99.6639% |
| Blingfire | 13.7092s | 1,000,001 | 99.9435% |
| **FlashBertTokenizer** | 5.6696s | 1,000,001 | 99.9484% |

### Motivation

- We need a tokenizer that is faster, more accurate, and easier to use than [Huggingface's BertTokenizerFast](https://github.com/huggingface/transformers/blob/main/src/transformers/models/bert/tokenization_bert_fast.py). ([link1](https://stackoverflow.com/questions/75595699/huggingfaces-berttokenizerfast-is-between-39000-and-258300-times-slower-than-ex), [link2](https://github.com/PaddlePaddle/PaddleNLP/issues/8565), [link3](https://blog.csdn.net/xhw205/article/details/129578988))
- [PaddleNLP's BertTokenizerFast](https://paddlenlp.readthedocs.io/en/stable/_modules/paddlenlp/experimental/faster_tokenizer.html) achieves a 1.2x performance improvement by implementing [Huggingface's Rust version](https://github.com/huggingface/tokenizers) in `C++`. However, using it requires installing both the massive [PaddlePaddle](https://github.com/PaddlePaddle/Paddle) and [PaddleNLP](https://github.com/PaddlePaddle/PaddleNLP) packages.
- [Tensorflow-text's FastBertTokenizer](https://www.tensorflow.org/text/api_docs/python/text/FastBertTokenizer) actually demonstrates slower performance in comparison.
- [Microsoft's Blingfire](https://github.com/microsoft/BlingFire) **takes over 8 hours** to train on custom data and shows relatively lower accuracy.
- [Rapid's cuDF](https://github.com/rapidsai/cudf) provides a GPU-based BertTokenizer, but it suffers from accuracy issues.
- Unfortunately, [FastBertTokenizer](https://github.com/georg-jung/FastBertTokenizer) and [BertTokenizers](https://github.com/NMZivkovic/BertTokenizers) developed in `C#` and cannot be used in `Python`.

### Your contribution

Although my Rust programming skills are limited, I'm willing to contribute to development if many people agree that optimizing BertTokenizerFast is necessary.