num_workers support (Multi-Core CPU)

Can the document splitting and other CPU centric functions be made multi-core ?

When running
required_exts = [".csv"]

reader = SimpleDirectoryReader(...

documents = reader.load_data(show_progress=True,num_workers=6)

ragatouille_pack = RAGatouilleRetrieverPack(
documents, # List[Document]
llm=Ollama(model="phi", request_timeout=10000.0),
index_name="my_index",
top_k=5
)

Start at the SimpleDirectoryReader.load_data entry point shown in the example and determine which document-splitting or other CPU-centric operations are in scope for num_workers. Check how show_progress and num_workers are currently handled, then define completion as safe multi-core execution for the requested operations with coverage for the example usage.

Written by the indexing model from the issue text.