How to allow the merging of consecutive newline tokens \n when training a byte-level bpe tokenizer?

Hello, I'm currently working on training a byte-level BPE tokenizer using the Huggingface tokenizers library. I've created a simple training script, a sample corpus, and provided the output produced by this script. My aim is to understand why consecutive newline tokens `\n` are not being merged into a single token `\n\n` during the tokenization process. Below are the details:

```python
from tokenizers import (
Tokenizer,
pre_tokenizers,
models,
decoders,
trainers,
processors,
)

files = ["demo_corpus.txt"]
tokenizer = Tokenizer(models.BPE())
tokenizer.pre_tokenizer = pre_tokenizers.Sequence([
pre_tokenizers.Digits(individual_digits=True),
pre_tokenizers.ByteLevel(add_prefix_space=False, use_regex=True)
])
tokenizer.decoder = decoders.ByteLevel()
tokenizer.post_processor = processors.ByteLevel()

trainer = trainers.BpeTrainer(
initial_alphabet=pre_tokenizers.ByteLevel.alphabet(),
vocab_size=2000,
special_tokens=[
"", "<|beginoftext|>", "<|endoftext|>"
]
)
tokenizer.train(files, trainer)
test_text = "#include \n\n\n\n\n"

print("pre-tokenize spans:", tokenizer.pre_tokenizer.pre_tokenize_str(test_text))
ids = tokenizer.encode(test_text).ids
print(f"tokens: {[tokenizer.decode([tid]) for tid in ids]}")
```

demo_corpus.txt:
```
#include

#include

#include

using namespace std;

int main(){
int N, A[100000], p = 0;

multiset S;

scanf("%d", &N);

int p0 = 0, q0 = 1, q = N-1;

vector result;

for(int i: result)

printf("%d\n", i);
}
```

output of training script:
```
pre-tokenize spans: [('#', (0, 1)), ('include', (1, 8)), ('Ġ<', (8, 10)), ('set', (10, 13)), ('>', (13, 14)), ('ĊĊĊĊĊ', (14, 19))]
tokens: ['#', 'include', ' <', 'set', '>', '\n', '\n', '\n', '\n', '\n']
```

the following is tokens produced by llama3 tokenizer:
```python
tokenizer = LlamaTokenizerFast.from_pretrained("my llama3 vocab path")
test_text = "#include \n\n\n\n\n"
print([tokenizer.decode([tid]) for tid in tokenizer(test_text)["input_ids"]])

# output
# ['<|begin_of_text|>', '#include', ' <', 'set', '>\n\n\n\n\n']
```