[Bug] Assert CompositeSpec order guarantees

Discussed in https://github.com/pytorch/rl/discussions/2341

^{Originally posted by hesic73 July 31, 2024}
I have a question about the order of keys returned by CompositeSpec.keys(include_nested=True, leaves_only=False). Is there any guarantee on the order of the keys, specifically that the iterator iterates from the deepest nodes to the root?

I want to write a function that removes all empty entries from a CompositeSpec:

def _remove_empty_entries(spec: CompositeSpec) -> CompositeSpec:
    for key in list(spec.keys(True, False)):
        if not isinstance(spec[key], CompositeSpec):
            continue
        if spec[key].is_empty():
            del spec[key]
    return spec

As shown in the function above, it assumes that the iterator processes from the deepest nodes to the root. This assumption is crucial for the function to work correctly.

If there is no such guarantee, how can I achieve this goal? A CompositeSpec with empty entries is quite undesirable.

Thank you!

Start at the CompositeSpec.keys(include_nested=True, leaves_only=False) entry point and inspect its traversal behavior. Define regression coverage for nested and empty entries based on the issue's deepest-to-root requirement, then run the existing CompositeSpec tests and verify that the asserted order supports removing empty composites.