kubeflow / kubeflow/docs-agent
bug(pipelines): ZeroDivisionError crash in chunk_and_embed when text splitter returns empty chunks
- Dominant language
- Python
- Stars
- 42
- Forks
- 111
- Avg merge
- 6d 23m
- Merged PRs (30d)
- 2
Description
## Bug Description
The `chunk_and_embed` component in `pipelines/kubeflow-pipeline.py` will
crash with a `ZeroDivisionError` if the `RecursiveCharacterTextSplitter`
returns an empty list for a given document.
## Affected Line
```python
print(f"File: {file_data['path']} -> {len(chunks)} chunks (avg: {sum(len(c) for c in chunks)/len(chunks):.0f} chars)")
```
## Root Cause
The pipeline already guards against short content with a `< 50` character
check before cleaning. However, the aggressive regex cleaning (Hugo
frontmatter removal, HTML tag stripping, URL removal) can reduce a
document that passed the 50-char check down to very little actual text.
The `RecursiveCharacterTextSplitter` then legitimately returns an empty
list, and the very next line divides by `len(chunks)` which is `0`,
crashing the entire pipeline component and failing the KFP run.
## Steps to Reproduce
1. Run the pipeline against a document that passes the `< 50` char guard
2. But whose content is entirely composed of Hugo frontmatter, HTML tags,
and URLs (all stripped by the cleaning regexes)
3. The splitter returns `chunks = []`
4. The print statement crashes with `ZeroDivisionError`
## Expected Behavior
Empty chunk lists should be gracefully skipped with a warning log,
identical to the existing `< 50` char guard pattern already in the code.
## Proposed Fix
```python
if not chunks:
print(f"Skipping file after chunking (no chunks produced): {file_data['path']}")
continue
```
I will submit a PR with this fix.
Contributor guide
Assessment
This issue has not been assessed yet.