llnl / llnl/Silo

integer data being compressed with shuffle filter

Open
#33 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
45
Forks
27
Avg merge
12h 34m
Merged PRs (30d)
4

Description

The shuffle filter in hdf5 is really for compressing floating point data a bit better. I don't think it helps much with integer data (though we should probably look into that also). It is nonetheless being applied in all cases.

-----------------------REDMINE MIGRATION-----------------------
This ticket was migrated from Redmine. As such, not all
information was able to be captured in the transition. Below is
a complete record of the original redmine ticket.

Ticket number: 1765
Status: New
Project: VisIt
Tracker: Bug
Priority: Normal
Subject: integer data being compressed with shuffle filter
Assigned to: -
Category: -
Target version: -
Author: Mark Miller
Start: 03/05/2014
Due date:
% Done: 0%
Estimated time:
Created: 03/05/2014 10:56 pm
Updated:
Likelihood: 3 - Occasional
Severity: 2 - Minor Irritation
Found in version: 4.8
Impact:
Expected Use:
OS: All
Support Group: Any
Description:
The shuffle filter in hdf5 is really for compressing floating point data a bit better. I don't think it helps much with integer data (though we should probably look into that also). It is nonetheless being applied in all cases.

Comments:

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the HDF5 shuffle-filter setup in the C compression or dataset-writing code and trace how the data type is identified. Check any existing compression tests, if present. Done means integer data no longer receives the shuffle filter while floating-point data retains the intended behavior, with tests covering both cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
c
Domain
database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.