python / python/typeshed

`ElementTree.parse` type hinted to return unusable generic types

Open
#14,672 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
5.1k
Forks
2.1k
Avg merge
1d 19h
Merged PRs (30d)
82

Description

Bug Report

Since upgrading to mypy 1.15+, using ElementTree has become increasingly difficult. Likely due to https://github.com/python/typeshed/pull/13349. I'm not sure if this extra strictness is expected or if I'm just holding it wrong.

One example is the ElementTree.parse function that now returns a generic ET.ElementTree[ET.Element[str]], which is no longer compatible with basic type signatures like ET.ElementTree, which is interpreted as ElementTree[Element[str] | None].

This adds many challenges that make it difficult to work with.

  • The generic argument here is very difficult to understand what it means and is lacking documentation. From what i've been able to deduce it's possible that the root node is of a different Element type or that the root node is None. While perhaps a valid scenario, it adds a big uphill to learning and usability.
  • If one were to use these generic types, typing out ET.ElementTree[ET.Element[str]] is a lot more verbose than ET.ElementTree, and i'm questioning if the strictness here is worth it.
  • If one tries to type hint the code with these generics. Python throws errors that 'ElementTree.Element' is not subscriptable. One workaround i found for this is to surround the type hints with double-quotes.

To Reproduce

import xml.etree.ElementTree as ET

def process_document(doc: ET.ElementTree) -> None:
    pass

def process_document2(doc: ET.ElementTree[ET.Element[str]]) -> None:    # TypeError: type 'xml.etree.ElementTree.Element' is not subscriptable
    pass

process_document(ET.parse("foo.xml"))     # mypy error: Argument 1 to "process_document" has incompatible type "ElementTree[Element[str]]"; expected "ElementTree[Element[str] | None]"  [arg-type]
process_document2(ET.parse("foo.xml"))

Run with mypy --strict

Expected Behavior

ET.parse returns types assignable to ET.ElementTree

Actual Behavior

Mypy returns errror

# mypy error: Argument 1 to "process_document" has incompatible type "ElementTree[Element[str]]"; expected "ElementTree[Element[str] | None]"  [arg-type]

Workarounds

  • Use ET.ElementTree().parse() function instead of ET.parse(), it returns the root-node as a type that can be assigned to ET.Element without any generic arguments.
    This is actually a pretty decent option, if this is considered to be the primary way of using ElementTree from now on, the documentation should reflect that, right now it's suggesting ET.parse()
  • Type out the complete "ElementTree[Element[str] | None]" wrapped with double-quotes.

Your Environment

  • Mypy version used:
  • Mypy command-line flags:
  • Mypy configuration options from mypy.ini (and other config files):
  • Python version used:

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading typeshed PR 13349 and the annotations for ElementTree.parse, then compare them with the ElementTree().parse() workaround and the Python ElementTree documentation. Done means the intended generic behavior and the expected assignability of ET.parse() are established, with the relevant typing or documentation issue addressed.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.