Unstructured-IO / Unstructured-IO/unstructured
bug/br tag tail text loss
Nobody has claimed this yet.
- Dominant language
- HTML
- Stars
- 15.5k
- Forks
- 1.3k
- Avg merge
- 4d 2h
- Merged PRs (30d)
- 13
Description
Describe the bug
The HtmlTable.from_html_text() method drops text content that follows <br/> tags when normalizing HTML tables. This causes loss of important content in table cells that contain line breaks.
To Reproduce
from unstructured.common.html_table import HtmlTable
html_text = """
<table>
<tr>
<td>This is 1st line.<br/>2nd line.<br/>3rd line.</td>
</tr>
</table>
"""
table = HtmlTable.from_html_text(html_text)
print(table.html)
Output:
<table><tr><td>This is 1st line.<br/><br/></td></tr></table>
Expected Output:
<table><tr><td>This is 1st line.<br/>2nd line.<br/>3rd line.</td></tr></table>
Expected behavior
The text content following <br/> tags should be preserved during HTML normalization. Currently, the tail text of <br/> elements is being removed, which results in loss of content.
Screenshots
No screenshots.
Environment Info
- unstructured version: 0.16.17
- Python version: 3.11
- OS: MacOS
Additional context
It is possible that the issue could be resolved by modifying the from_html_text() method to preserve the tail text of <br/> tags while normalising whitespace.
class HtmlTable:
...
@classmethod
def from_html_text(cls, html_text: str) -> 'CustomHtmlTable':
...
# -- normalize br tag tail text
if e.tag == "br":
if e.tail:
e.tail = " ".join(e.tail.split())
else:
# -- remove tails for non-br elements
if e.tail:
e.tail = None
...
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at HtmlTable.from_html_text() and run the reproduction with the provided table containing br tags. Trace how br tail text is normalized, then verify that the resulting table.html preserves the text after each line break and matches the expected output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 65/100