datastore: block flatfs has too many buckets.

Version/Platform/Processor information: 0.4.2/Windows 10 x64

Type: bug/enchancement
Area: blockstore
Priority: P0/P1

Description:
The current flatdb format for blocks has way too many buckets. Most filesystems slow down drastically as the number of objects in a directory increases.

Git uses 2 hexadecimal characters for buckets. I believe 2 or 3 is the most a filesystem like Windows's NTFS can handle without drastic slowdown.
Alternatively, the blockdb could use several directories, as in blocks/1a/2b/3c4d5e...9f.data

When adding a lot of objects, for example a 20 gigabyte file, in the current system of 8 characters per bucket, there is an average of 2 .data files per bucket, and the amount of buckets themselves slow down block operations by at least a factor of 8 (NTFS)

---

Note that I didn't mark the title with the issue type. That is because I'm not quite sure what type of issue this is; I consider it a suggestion.

Start by locating the blockstore's flatdb layout and measure how its current bucket structure behaves on Windows/NTFS. Evaluate the proposed shorter hexadecimal buckets or nested directories; done means selecting a layout that avoids the reported slowdown while continuing to store blocks correctly.

Written by the indexing model from the issue text.