Proposal: tiny failure-mode test pack for docling chunk expansion (from WFGY Problem Map)

Hi Major,

first of all, thanks for starring my WFGY repo – it has been sitting in your stars list for a while and that really helped me keep going.

I am the author of WFGY 3.0 (now I have 1.0 + 2.0 +3.0 ),
which is basically a “problem map” of 131 failure modes for LLM pipelines (retrieval, planning, infra, etc.).
The first few entries are exactly about things like:

• hallucination + chunk drift
• list items being cut away from their parents
• debugging retrieval as a black box

When I read this repo’s README:

The goal of this script is to take a chunk of text from a docling document and work back to the section header of the document that contains the chunk. This would allow us to expand the chunk with additional context from the document.

I immediately thought: this is basically one very concrete slice of my ProblemMap No.1 / No.2 / No.8.

So I wanted to ask if a very small collaboration would be useful to you:

1. I prepare a tiny set of adversarial doc fragments (in the spirit of WFGY ProblemMap)
   • list items partially cut from their parent section
   • code blocks inside lists
   • weird header nesting that tends to confuse chunk → parent mapping
2. I wrap a minimal script / Colab that runs docling-expander on those cases
   • no changes to Docling itself, just calling your existing main.py
   • the script reports: expected parent header vs. what Docling actually finds
3. Optionally I add a column that tags each failure with a ProblemMap code
   (for example “No.1 hallucination & chunk drift”, “No.2 interpretation collapse”, etc.)

The goal would not be to claim “WFGY fixes Docling”.
It is more like: “can we get a clearer picture of how chunk → parent mapping breaks, using a structured failure taxonomy that is already public and fully auditable?”

If that sounds interesting, I am happy to:

• keep the deliverable extremely small (a few text files + one script / Colab)
• send it as a PR or just a Gist / notebook, whichever fits your workflow better
• and of course I can share the ProblemMap overview here:
  • https://github.com/onestardao/WFGY/tree/main/ProblemMap/README.md

Also, if you ever want to test IBM watsonx (or any other models) on more “out-of-distribution / stability” kind of questions, I would love to plug a tiny subset of the WFGY 3.0 S-class pack into modelgrader as a special suite.
No pressure at all – just an offer if that happens to align with your current experiments.

Either way, thanks again for the star and for all the work you are doing around RAG and LLM evaluation.
Your repos have been very helpful references for me while building WFGY.

Start by reading the README and the existing main.py entry point to understand how docling-expander maps chunks to parent headers. Prepare a few adversarial document fragments and a small script or Colab that records the expected parent header alongside the result from the existing tool. Done means the failure cases and comparison output are reproducible without changing Docling.