kubeflow / kubeflow/docs-agent

bug(pipelines): ZeroDivisionError in chunk_and_embed_incremental when text splitter returns empty chunks

Open
#163 1 comment 0 reactions 1 assignee Claimed by @Kunal-Somani View on GitHub
Dominant language
Python
Stars
42
Forks
111
Avg merge
6d 23m
Merged PRs (30d)
2

Description

## Bug Description

`chunk_and_embed_incremental` in `pipelines/incremental-pipeline.py` crashes with `ZeroDivisionError` when `RecursiveCharacterTextSplitter` returns an empty chunk list.

PR #161 fixed the identical bug in `kubeflow-pipeline.py` but `incremental-pipeline.py` has the same crash on line 223 and remains unaddressed.

## Affected Line
```python
print(f"File: {file_data['path']} -> {len(chunks)} chunks (avg: {sum(len(c) for c in chunks)/len(chunks):.0f} chars)")
```

## Root Cause

Same as #148 — aggressive regex cleaning can reduce a document that passed the `< 50` char guard to nothing after splitting. The splitter returns `chunks = []` and the print statement divides by zero, crashing the entire incremental KFP run.

## Proposed Fix
```python
if not chunks:
print(f"Skipping file after chunking (no chunks produced): {file_data['path']}")
continue
```

I will submit a PR with this fix.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.