Unstructured-IO / Unstructured-IO/unstructured

bug/br tag tail text loss

Open Beginner friendly
#3,899 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
HTML
Stars
15.5k
Forks
1.3k
Avg merge
4d 2h
Merged PRs (30d)
13

Description

Describe the bug
The HtmlTable.from_html_text() method drops text content that follows <br/> tags when normalizing HTML tables. This causes loss of important content in table cells that contain line breaks.

To Reproduce

from unstructured.common.html_table import HtmlTable
html_text = """
<table>
<tr>
<td>This is 1st line.<br/>2nd line.<br/>3rd line.</td>
</tr>
</table>
"""
table = HtmlTable.from_html_text(html_text)
print(table.html)

Output:

<table><tr><td>This is 1st line.<br/><br/></td></tr></table>

Expected Output:

<table><tr><td>This is 1st line.<br/>2nd line.<br/>3rd line.</td></tr></table>

Expected behavior

The text content following <br/> tags should be preserved during HTML normalization. Currently, the tail text of <br/> elements is being removed, which results in loss of content.

Screenshots
No screenshots.

Environment Info

  • unstructured version: 0.16.17
  • Python version: 3.11
  • OS: MacOS

Additional context
It is possible that the issue could be resolved by modifying the from_html_text() method to preserve the tail text of <br/> tags while normalising whitespace.

class HtmlTable:
    ...
    @classmethod
    def from_html_text(cls, html_text: str) -> 'CustomHtmlTable':
            ...
            # -- normalize br tag tail text
            if e.tag == "br":
                if e.tail:
                    e.tail = " ".join(e.tail.split())
            else:
                # -- remove tails for non-br elements
                if e.tail:
                    e.tail = None
            ...

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at HtmlTable.from_html_text() and run the reproduction with the provided table containing br tags. Trace how br tail text is normalized, then verify that the resulting table.html preserves the text after each line break and matches the expected output.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.