gonum / gonum/hdf5

Pure Go HDF5 Reader Library - Complementary Alternative for Cross-Platform Deployments

Open
#86 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
135
Forks
34
PR merge metrics
No merged PRs in 30d

Description

Hi HDF5 community! 👋

I wanted to share a new pure Go HDF5 library that might be useful for certain use cases: **[scigolib/hdf5](https://github.com/scigolib/hdf5)**

## Background

Your `gonum/hdf5` library has been an excellent reference and inspiration! While it provides comprehensive HDF5 functionality through CGo bindings, we've built a **pure Go read-only implementation** to address specific
deployment scenarios.

## Why Pure Go?

We created this library to solve challenges with:
- **Cross-compilation** - No C dependencies = easier builds for multiple platforms
- **Containerization** - Smaller Docker images without HDF5 C libraries
- **Cloud deployments** - Simpler deployment on AWS Lambda, Cloud Functions, etc.
- **Windows/macOS** - No need to install HDF5 system libraries

## Current Status (v0.9.0-beta)

**Supported:**
- ✅ Superblock versions 0, 2, 3
- ✅ Object headers (v2)
- ✅ Groups (traditional + modern formats)
- ✅ Datasets (compact, contiguous, chunked with GZIP)
- ✅ Datatypes (integers, floats, strings, compounds)
- ✅ File traversal and navigation
- ✅ Cross-platform CI/CD (Linux, macOS, Windows)

**Not Yet:**
- ❌ Writing (read-only by design for safety)
- ❌ Advanced compression (SZIP, LZF, etc.)
- ❌ Some complex datatypes (arrays, enums, references)

## Use Cases

**Use `gonum/hdf5` when you need:**
- Full read/write capabilities
- Advanced HDF5 features
- Maximum compatibility

**Use `scigolib/hdf5` when you need:**
- Pure Go (no CGo)
- Easy cross-compilation
- Lightweight deployments
- Read-only access is sufficient

## Feedback Welcome!

We're actively testing against real-world HDF5 files. If anyone has:
- Edge cases or problematic files to test
- Suggestions for priority features
- Interest in collaboration

Please feel free to open issues or discussions!

## Links

- **Repository:** https://github.com/scigolib/hdf5
- **Module:** `github.com/scigolib/hdf5`
- **License:** MIT

Thank you to the `gonum/hdf5` maintainers for the excellent reference implementation! 🙏

---

*Note: This is not meant to replace `gonum/hdf5` but to complement it for specific deployment scenarios. Both libraries serve different needs.*

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue contains no repository files, tests, or entry points to inspect. It announces the external scigolib/hdf5 project and requests feedback rather than defining a change or completion criteria, so there is no actionable starting point or clear definition of done.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
data
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
10/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.