llnl / llnl/Silo

Add support for HDF5 compact storage

Open
#62 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
45
Forks
27
Avg merge
12h 34m
Merged PRs (30d)
4

Description

I tested this in a hacked version of Silo but discovered that if we use COMPACT storage mode for a lot of the smaller datasets a silo application generates, we can really reduce metadata thrashing. I would have put this into the 4.8 release but did not understand the implications enough to feel comfortable doing so. To actually enable this capability is dirt simple. However, to fully test it and see if it effects performance/diffing in some strange ways, will take some effort.

-----------------------REDMINE MIGRATION-----------------------
This ticket was migrated from Redmine. As such, not all
information was able to be captured in the transition. Below is
a complete record of the original redmine ticket.

Ticket number: 352
Status: New
Project: VisIt
Tracker: Feature
Priority: Normal
Subject: Add support for HDF5 compact storage
Assigned to: -
Category: -
Target version: -
Author: Mark Miller
Start: 08/19/2010
Due date:
% Done: 0%
Estimated time: 8.00 hours
Created: 08/19/2010 03:42 pm
Updated:
Likelihood:
Severity:
Found in version:
Impact: 4 - High
Expected Use: 4 - Common
OS: All
Support Group: Any
Description:
I tested this in a hacked version of Silo but discovered that if we use COMPACT storage mode for a lot of the smaller datasets a silo application generates, we can really reduce metadata thrashing. I would have put this into the 4.8 release but did not understand the implications enough to feel comfortable doing so. To actually enable this capability is dirt simple. However, to fully test it and see if it effects performance/diffing in some strange ways, will take some effort.

Comments:

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are named. Start by locating Silo's handling of dataset storage modes and existing tests for metadata, performance, or diffing; then verify compact storage for smaller datasets and check that behavior and comparisons remain correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
databases, performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.