Improve Silo Block Based VFD to 7z-compress the blocks

This is harder than it sounds. When blocks are compressed, their offest in the file is no longer easily predictable. So, you need to store the blockoffsets or offset deltas too. Either allocate enough space at the front of the file to store all block offsets for the file (would require foreknowledge of approx. file size), or develop a scheme of storing the block offset data among other data blocks in the file. But, keeping the blocks 7z compressed would alleviate overheadheavy scenarios.

-----------------------REDMINE MIGRATION-----------------------
This ticket was migrated from Redmine. As such, not all
information was able to be captured in the transition. Below is
a complete record of the original redmine ticket.

Ticket number: 3186
Status: New
Project: VisIt
Tracker: Feature
Priority: Normal
Subject: Improve Silo Block Based VFD to 7z-compress the blocks
Assigned to: -
Category: -
Target version: -
Author: Mark Miller
Start: 07/26/2018
Due date:
% Done: 0%
Estimated time:
Created: 07/26/2018 10:38 pm
Updated:
Likelihood:
Severity:
Found in version:
Impact: 3 - Medium
Expected Use: 3 - Occasional
OS: All
Support Group: Any
Description:
This is harder than it sounds. When blocks are compressed, their offest in the file is no longer easily predictable. So, you need to store the blockoffsets or offset deltas too. Either allocate enough space at the front of the file to store all block offsets for the file (would require foreknowledge of approx. file size), or develop a scheme of storing the block offset data among other data blocks in the file. But, keeping the blocks 7z compressed would alleviate overheadheavy scenarios.

Comments:

No source file, test, or entry point is named. Start by locating Silo's block-based VFD implementation and its existing block metadata handling, then determine how compressed block offsets could be stored. Done means block data is 7z-compressed while reads can reliably locate and decompress every block.

Written by the indexing model from the issue text.