Recursive list of medium-sized bucket is 55x slower than s3cmd

A simple `s4cmd ls -r s3://[bucket-name]` on a bucket with about 40,000 objects evenly distributed across about 100 top-level prefixes takes over 10 minutes (at the default 16 threads), while the exact same `s3cmd ls -r s3://[bucket-name]` takes only about 12 seconds.

I understand that multithreading adds some overhead for small buckets like this, but I would not have anticipated such a massive downgrade in performance. A _non_-recursive `ls` on the same bucket (with ~100 results) takes <1s for both s3cmd and s4cmd.

What could account for such a massive differential?

Reproduce the `s4cmd ls -r s3://[bucket-name]` comparison against `s3cmd` using a bucket with about 40,000 objects across roughly 100 prefixes and the default 16 threads. Trace the recursive listing entry point and measure where time is spent relative to non-recursive `ls`. Done means identifying the cause of the slowdown and documenting a verified performance improvement.

Written by the indexing model from the issue text.