Can i use this script to pre-train models?

Hi there!
I am currently working on pre-training a Llama or Mistral model with clinical texts. Is there any way to use this QLoRA + FSDP script to do such training? Should I make any changes to the current code to be able to do pre-training?

Start by reading the QLoRA + FSDP script and its documented training flow, then compare the stated pre-training goal for Llama or Mistral models with the current scope. Done would require a maintainer-confirmed path for pre-training on clinical texts and clear requirements for any code changes.

Written by the indexing model from the issue text.