CSV and JSON knowledge sources embed the repr of the content dict instead of the file text

Description

CSVKnowledgeSource and JSONKnowledgeSource chunk str(self.content) — the Python repr of the whole dict[Path, str] — instead of each file's text. Every chunk that gets embedded and stored therefore carries the PosixPath('...') wrapper, quotes and escaped newlines, and a multi-file source has all of its files merged into one string.

# lib/crewai/src/crewai/knowledge/source/csv_knowledge_source.py:30  (same in aadd() at :37)
# lib/crewai/src/crewai/knowledge/source/json_knowledge_source.py:42 (same in aadd() at :49)
content_str = (
    str(self.content) if isinstance(self.content, dict) else self.content
)
new_chunks = self._chunk_text(content_str)

BaseFileKnowledgeSource declares content: dict[Path, str] = Field(init=False, default_factory=dict) and its model_post_init runs self.content = self.load_content() unconditionally, so for these two sources the isinstance(..., dict) branch is not a corner case — it is the only branch that ever runs (content cannot even be passed to __init__).

Every other file knowledge source in the same package already iterates the mapping instead: text_file_knowledge_source.py:26/:33, pdf_knowledge_source.py:46/:53, excel_knowledge_source.py:151/:165. CSV and JSON are the only two outliers, and after the fix str(self.content) no longer appears anywhere under lib/crewai/src/crewai/knowledge/.

Steps to Reproduce

from pathlib import Path
from unittest.mock import MagicMock

from crewai.knowledge.source.csv_knowledge_source import CSVKnowledgeSource

path = Path("knowledge/data.csv")
path.parent.mkdir(exist_ok=True)
path.write_text("Name,Age\nBrandon,30\nAlice,25\n", encoding="utf-8")

source = CSVKnowledgeSource(file_paths=[path])
source.storage = MagicMock()
source.add()
print(source.chunks)

Actual behavior (measured on main @ 3831e8b)

["{WindowsPath('C:/.../knowledge/data.csv'): 'Name Age\\nBrandon 30\\nAlice 25\\n'}"]

With {"name": "Brandon", "age": 30} through JSONKnowledgeSource:

["{WindowsPath('C:/.../knowledge/data.json'): 'name: Brandon\\nage: 30\\n'}"]

JSONKnowledgeSource._json_to_text() exists precisely to turn the document into readable key: value text — that work is thrown away one line later by re-escaping it into a repr.

Expected behavior

The same as TextFileKnowledgeSource handed the identical text:

['Name Age\nBrandon 30\nAlice 25\n']

one chunk group per file, containing only that file's text.

Impact

• Embeddings for every CSV/JSON source are computed over path prefixes, quotes and \\n literals, so retrieval quality is degraded by markup no user put in the file.
• Multi-file sources lose per-file chunking: file A's tail and file B's head land in the same chunk, and the chunk_size / chunk_overlap window slides over repr offsets instead of text offsets.
• Absolute file paths are written into stored chunks, so they end up in the context sent to the model.

Possible Solution

Match the sibling sources — chunk each file's text on its own, in both add() and aadd():

for text in self.content.values():
    new_chunks = self._chunk_text(text)
    self.chunks.extend(new_chunks)
self._save_documents()          # await self._asave_documents() in aadd()

Additional context

The reason this went unnoticed is that no test drives add()/aadd() on these two sources: test_csv_knowledge_source and test_json_knowledge_source patch the vector DB and assert on the query path, so the corrupted chunk content is never inspected. test_file_path_validation and the storage assertions would both catch it once such a test exists.

Authored with an AI coding assistant. .github/CONTRIBUTING.md requires the llm-generated label for agent-authored contributions; external contributors cannot apply labels here, so maintainers please add it.

Operating System

Windows 11 (nothing OS specific — WindowsPath in the output is PosixPath on Linux)

Python Version

3.13

crewAI Version

1.15.22 (main @ 3831e8b)

crewAI Tools Version

Not involved — the defect is in the crewai package.

Virtual Environment

Venv

Start with lib/crewai/src/crewai/knowledge/source/csv_knowledge_source.py and json_knowledge_source.py, then compare their add() and aadd() methods with text_file_knowledge_source.py, pdf_knowledge_source.py, and excel_knowledge_source.py. Add or update coverage in test_csv_knowledge_source and test_json_knowledge_source to inspect chunk contents for single- and multi-file inputs. Done means each file's text is chunked separately without path wrappers or escaped newlines.