implement dynamic chunking

Need advanced chunking methods: Chunk size is dynamic depending on the content, and can overload the model if the chunk size is too large.
Use semantic chunking to chunk the data into smaller pieces based on semantic meaning.

Look for existing chunking logic in the codebase, likely in a preprocessing or data loading module. Research semantic chunking techniques in LangChain's documentation. The goal is to implement a method that adjusts chunk size based on content semantics to prevent model overload. Testing will involve processing sample data and verifying chunk sizes are appropriate.

Escrita pelo modelo de indexação a partir do texto da issue.