How to continue to train ModernBert base/large in a specific domain?

Hi, thanks for the wonderful work!
I was trying to continue training Modernbert in a specific domain. I could not find detailed documentations, would you please help me to answer some of the questions I have.
This is what I have done:
1. I'm using 8-A100 GPUs, and the total domain texts have about 1 billion tokens.
2. I created conda env based on the environment.yaml and installed "flash_attn==2.6.3" --no-build-isolation.
3. I'm using the pretraining_documentation branch, the modernbert-base-pretrain.yaml (https://github.com/AnswerDotAI/ModernBERT/tree/pretraining_documentation/yamls/modernbert#:~:text=2%20months%20ago-,modernbert%2Dbase%2Dpretrain.yaml,-Update%20%26%20add%20ModernBERT).
4. Also, I'm trying to use the checkpoint to continue to train, found here: https://huggingface.co/answerdotai/ModernBERT-base-training-checkpoints/tree/main

My questions are:
1. My data are in csv file with one row of an article, should I split each article into sentences, or it will be split automatically with max_seq_len of 1024? Since the corpus are not large, I can convert to MDS format and save it locally, and set streaming to False.
2. How should I split the corpus into training and validation?
3. Which checkpoint I need to use from the huggingface hub, there are three folders: pretrain, learning-rate-decay, and context-extension?
4. Where should I specify the checkpoint in the yaml file? is it in the bottom with 'load_path'?
Thank you!

Start with the pretraining_documentation branch, environment.yaml, and yamls/modernbert/modernbert-base-pretrain.yaml. Review the referenced Hugging Face checkpoint folders and the existing training configuration, then document the data preparation, train/validation split, checkpoint selection, and load_path guidance requested in the issue. Done means the questions have clear, reproducible answers for this training setup.

Written by the indexing model from the issue text.