Downloading taskonomy with download_tnb.sh

I am having trouble downloading the taskonomy dataset using download_tnb.sh. I created a directory for each of the buildings and ran the file. Now, my files look like this (I will give one filename as example):
dataset/
----benevolence/
--------class_object/
------------point_0_view_0_domain_class_object.npy
--------class_scene/
------------point_0_view_0_domain_class_scene.npy
--------normal/
------------point_0_view_0_domain_normal.npy
--------rgb/
------------point_0_view_0_domain_rgb.png
----other_building/
etc

With this file structure, generate_splits.py looks at wrong files and my splits are empty, for example benevolence.json has "[]" an empty list as entry.

I have been struggling to find the right way to download the data with the correct naming and could really use some help.

Start with download_tnb.sh and generate_splits.py, then compare the generated directory and file names with the paths generate_splits.py scans. Reproduce the empty benevolence.json result using the shown dataset layout; done means the downloader and split generator agree and produce non-empty splits.

Written by the indexing model from the issue text.