python / python/cpython

Slight ElementTree serialization performance enhancement for trees with str tags

Open Beginner friendly
#118,687 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

type-feature
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Feature or enhancement

Proposal:

This proposal improves the performance of writing xml whose trees are made up of tag names that are predominantly strings. This comes at the cost of performance for trees with tags that are predominantly QNames

As far as I'm aware, using a str for the tag name is more common than using a QName and we should optimise for that scenario (for example, parsing an xml document with ElementTree returns Elements whose tags are all strings).

Reordering the following if block to make the isinstance(tag, str) check first gives a performance improvement of 1 - 1.5% on a tree parsed from a file that was about 300kb:

--- a/Lib/xml/etree/ElementTree.py
+++ b/Lib/xml/etree/ElementTree.py
@@ -827,12 +827,12 @@ def add_qname(qname):
     # populate qname and namespaces table
     for elem in elem.iter():
         tag = elem.tag
-        if isinstance(tag, QName):
-            if tag.text not in qnames:
-                add_qname(tag.text)
-        elif isinstance(tag, str):
+        if isinstance(tag, str):
             if tag not in qnames:
                 add_qname(tag)
+        elif isinstance(tag, QName):
+            if tag.text not in qnames:
+                add_qname(tag.text)
         elif tag is not None and tag is not Comment and tag is not PI:
             _raise_serialization_error(tag)
         for key, value in elem.items():

As this enhancement is within a loop that traverses the entire xml document, the larger the xml tree, the greater the performance improvement as the tree traversal starts to account for more time than other setup code.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Open Lib/xml/etree/ElementTree.py and inspect add_qname's elem.iter() loop. Check the proposed branch ordering for string and QName tags, then verify XML serialization remains correct and compare performance on a parsed tree similar to the reported 300kb example. Done means the string-tag case improves without breaking QName or invalid-tag handling.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
65/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.