NVIDIA / NVIDIA/NeMo-Retriever

[FEA]: docx improvements

Open
#316 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature request
Dominant language
Python
Stars
3k
Forks
349
Avg merge
1d 23h
Merged PRs (30d)
116

Description

Is this a new feature, an improvement, or a change to existing functionality?

New Feature

How would you describe the priority of this feature request

Would be nice

Please provide a clear description of problem this feature solves

Hi,

A few improvements can be made to docx parsing in docxreader.py

  1. improve management of bullet lists
  2. fix bug with table_images
  3. handle headers and footers
Describe the feature, and optionally a solution or implementation and any alternatives
  1. improve management of bullet lists in apply_paragraph_style (some bullet lists are not detected by style.startswith("List")):
        try:
            numPr = paragraph._element.xpath("./w:pPr/w:numPr")[0]
            level = int(numPr.xpath("./w:ilvl/@w:val")[0])
        except Exception:
            numPr = None
            level = 0
        style = paragraph.style.name

        # Apply style
        if re.match(r"^Heading [1-9]$", style):
            n = int(style.split(" ")[-1])
            text = f"{'#' * n} {text}"
        elif style.startswith("List") or (numPr is not None):
               ...
  1. table images are not well formatted
from itertools import chain
...
        table_images = [cell_images for row in rows for _, cell_images in row]
        table_images = list(chain(*chain(*table_images)))

instead of

table_images = [image for row in rows for _, images in row for image in images]
  1. the current version does not handle headers and footers

To do so, you can replace lin 219

    for c in paragraph.iter_inner_content():
        ...

by

        for section in self.document.sections:
              self.text += "".join([self.format_paragraph(p)[0] + "\n" for p in section.header.paragraphs])

              for c in section.iter_inner_content():
                  ...

              self.text += "".join([self.format_paragraph(p)[0] + "\n" for p in section.footer.paragraphs])
Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Open docxreader.py and inspect apply_paragraph_style plus the code around line 219; compare the current bullet-list and table-image handling with the examples in the issue. Done means DOCX parsing correctly recognizes the listed bullet styles, preserves table images, and includes section headers and footers.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
content
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.