merge_index doesn't show path with subfolders

Hi!
I'm trying to merge index.jsons into one, so I have folder

dataset/
    part.00000/
      train/
        index.json
        shard.00000.mds
        …
      val/
        index.json
        shard.00000.mds
        …
    part.00001/
      train/
        index.json
        shard.00000.mds
        …
      val/
        index.json
        shard.00000.mds
        …
    …

after

from streaming.base.util import merge_index
merge_index('dataset', keep_local=True)

I'm getting

dataset/
    index.json <- created afterward by merging the  index files
    part.00000/
      train/
        index.json
        shard.00000.mds
        …
      val/
        index.json
        shard.00000.mds
        …
    part.00001/
      train/
        index.json
        shard.00000.mds
        …
      val/
        index.json
        shard.00000.mds
        …
    …

But I want to verify, how new index.json will know about subfolder information?
If i'm using

import json

with open('/dataset/index.json') as f:
    d = json.load(f)


print(d['shards'][1400]) # 7th file of part0001/train/
print(d['shards'][7]) # 7th file of part0000/train/

I will have both with 'basename': 'train/shard.00007.mds' but in real it's are two different files: 'part.0000/train/shard.00007.mds' and 'part.0001/train/shard.00007.mds'

{'column_encodings': ['bytes'], 'column_names': ['tokens'], 'column_sizes': [None], 'compression': 'zstd', 'format': 'mds', 'hashes': [], 'raw_data': {'basename': 'train/shard.00007.mds', 'bytes': 67092639, 'hashes': {}}, 'samples': 4093, 'size_limit': 67108864, 'version': 2, 'zip_data': {'basename': 'train/shard.00007.mds.zstd', 'bytes': 16269680, 'hashes': {}}}
{'column_encodings': ['bytes'], 'column_names': ['tokens'], 'column_sizes': [None], 'compression': 'zstd', 'format': 'mds', 'hashes': [], 'raw_data': {'basename': 'train/shard.00007.mds', 'bytes': 67092639, 'hashes': {}}, 'samples': 4093, 'size_limit': 67108864, 'version': 2, 'zip_data': {'basename': 'train/shard.00007.mds.zstd', 'bytes': 16187752, 'hashes': {}}}

I want to verify correctness

Start at streaming.base.util.merge_index and inspect how it builds the merged index from nested part.00000/train and part.00001/train directories. Compare the generated index.json with the source shard locations shown in the issue. Done means each merged shard entry preserves a path that uniquely identifies its original part and subfolder.