kubeflow / kubeflow/docs-agent

bug(pipelines): ZeroDivisionError crash in chunk_and_embed when text splitter returns empty chunks

Open
#148 1 comment 0 reactions 1 assignee Claimed by @Kunal-Somani View on GitHub
Dominant language
Python
Stars
42
Forks
111
Avg merge
6d 23m
Merged PRs (30d)
2

Description

## Bug Description

The `chunk_and_embed` component in `pipelines/kubeflow-pipeline.py` will
crash with a `ZeroDivisionError` if the `RecursiveCharacterTextSplitter`
returns an empty list for a given document.

## Affected Line
```python
print(f"File: {file_data['path']} -> {len(chunks)} chunks (avg: {sum(len(c) for c in chunks)/len(chunks):.0f} chars)")
```

## Root Cause

The pipeline already guards against short content with a `< 50` character
check before cleaning. However, the aggressive regex cleaning (Hugo
frontmatter removal, HTML tag stripping, URL removal) can reduce a
document that passed the 50-char check down to very little actual text.
The `RecursiveCharacterTextSplitter` then legitimately returns an empty
list, and the very next line divides by `len(chunks)` which is `0`,
crashing the entire pipeline component and failing the KFP run.

## Steps to Reproduce

1. Run the pipeline against a document that passes the `< 50` char guard
2. But whose content is entirely composed of Hugo frontmatter, HTML tags,
and URLs (all stripped by the cleaning regexes)
3. The splitter returns `chunks = []`
4. The print statement crashes with `ZeroDivisionError`

## Expected Behavior

Empty chunk lists should be gracefully skipped with a warning log,
identical to the existing `< 50` char guard pattern already in the code.

## Proposed Fix
```python
if not chunks:
print(f"Skipping file after chunking (no chunks produced): {file_data['path']}")
continue
```

I will submit a PR with this fix.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.