gazebosim / gazebosim/sdformat
XSD 1.0 can't efficiently represent SDF
- Dominant language
- C++
- Stars
- 216
- Forks
- 125
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 14
Description
I found another issue with the XSD generator that also exists in the python version. The short version of it is that the current XSD will ignore the `required` attribute of SDF and instead turn every attribute into `required="*"`.
The more detailed explanation is that the generator aggregates all elements of a ComplexType (an element with children) inside a `` element. `maxOccurs` refers to the number of occurrences of this block within the _parent_, meaning that - for an infinite number of times - you get to choose any one child element listed inside ``. This effectively allows any element to occur an infinite number of times, and (assuming there is more than one element inside choice) makes every element optional.
XSD 1.0 doesn't offer any good solution.
The only one that I could come up with (that doesn't involve changing SDF) is the following: Whenever an element contains unbounded elements, gather all bounded elements and put them into a sequence (fixed order). Then, for each permutation of that sequence insert the unbounded elements as an optional choice between them (new, longer sequence), and append the long sequence into one massive choice block. If you prefer this in code I put the snippet below
snippet
```python
# sort all elements into two categories:
# 1. maxOccurs <= 1
# 2. maxOccurs = unbounded and minOccurs = 0
# the case maxOccurs = unbounded and minOccurs = 1 is refactored
# into 1 required element in case 1 + 1 optional element in case 2
bounded_elements = [el for el in elements if not el.attrib["maxOccurs"] == "unbounded"]
optional_unbounded = list()
for unbounded_element in [el for el in elements if el.attrib["maxOccurs"] == "unbounded"]:
unbounded_element:ElementTree.Element
min_occurs = unbounded_element.attrib["minOccurs"]
if min_occurs == "1":
required_item = deepcopy(unbounded_element)
required_item.set("maxOccurs", "1")
bounded_elements.append(required_item)
# it will be ugly, but we can try to reduce clutter
unbounded_element.attrib.pop("minOccurs")
unbounded_element.attrib.pop("maxOccurs")
optional_unbounded.append(unbounded_element)
infinity_choice = ElementTree.Element(_to_qname("xs:choice"))
infinity_choice.set("maxOccurs", "unbounded")
infinity_choice.extend(optional_unbounded)
# bounded elements may show up anywhere between unbounded optional elements.
# However we can't use here (XSD 1.0 limitation). Instead use a choice
# over sequences of unbounded choices with a permutation of bounded elements
# inbetween them.
container = ElementTree.Element(_to_qname("xs:choice"))
for sequence in permutations(bounded_elements):
seq_container = ElementTree.Element(_to_qname("xs:sequence"))
seq_container.append(infinity_choice)
for pair in zip_longest(sequence, [], fillvalue=infinity_choice):
seq_container.extend(pair)
container.append(seq_container)
el.append(container)
el.extend(attributes)
```
This accurately represents the `required` attributes of SDF at the cost of generating more complex XSD. The problem with this solution is that some elements have a lot of children. For example world.sdf has 10 bounded elements meaning that we have to generate a choice block containing 1024 sequences. This doesn't sound like a lot, but I aborted the generated XSD at 3.5GB file size...
Another solution is to change/restrict SDF and enforce order among child elements, e.g., only allow elements to occur in the order in which they are listed inside `X.sdf`, or enforce that bounded elements (requires=0 or requires=1) always occur before any unbounded elements (requires=* or requires=+). Anything that reduces the number of permutations that could occur.
A third solution is to switch/upgrade to XSD 1.1 which allows us to use `maxOccurs="unbounded"` inside a `` block, making the entire problem obsolete. The downside is that XSD 1.1 isn't as widespread as XSD 1.0 which - for example - means that `xmllint` can not validate it anymore.
I'm not sure which approach is the desired one here; @azeey do you have any thoughts/comments?
Contributor guide
Research direction
The issue names no implementation file or test. Start by locating the XSD generator and its Python counterpart, then inspect generated output for world.sdf and the source ordering in X.sdf. Before coding, resolve whether to target XSD 1.0, constrain SDF ordering, or adopt XSD 1.1; done means a chosen approach preserves required attributes without impractical schema growth.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python, xml
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100