HDFGroup / HDFGroup/hdfview

Feature Request: Floating-Point Display Precision Control

Open
#408 0 comments 0 reactions 1 assignee Assigned to @mattjala View on GitHub
Component - HDFView
Dominant language
Java
Stars
153
Forks
26
Avg merge
47m
Merged PRs (30d)
1

Description

# Feature Request: Floating-Point Display Precision Control

## Summary
Add user-configurable precision control for floating-point number display in HDFView, allowing users to control how many decimal places are shown.

## Motivation
Currently, HDFView displays floating-point values with full precision, which can lead to:
- Visual clutter with values like `2.0999999999999996` instead of `2.1`
- Difficulty comparing values visually
- Confusion about actual data vs. display artifacts

Many scientific applications allow users to control display precision separate from the underlying data precision.

## Proposed Solution

### 1. Add Precision Setting to User Options

**Location**: User Preferences → HDF Options (UserOptionsHDFPage.java)

**New Setting**:
- **Name**: "Floating-point display precision"
- **Type**: Integer spinner (range: 1-16, default: 6)
- **Property key**: `ViewProperties.DATA_DISPLAY_PRECISION`

### 2. Display Formatting

Apply precision when formatting numbers for display:

```java
// Example implementation in DataDisplayConverterFactory
String format = String.format("%%.%df", precision);
displayValue = String.format(format, doubleValue);
```

### 3. User Interface

Add to HDF Options dialog:
```
┌─ Display Options ────────────────────┐
│ │
│ [✓] Show scientific notation │
│ │
│ Floating-point precision: [6 ▼] │
│ (Number of decimal places: 1-16) │
│ │
└──────────────────────────────────────┘
```

## Benefits

1. **Better readability**: Users can hide insignificant digits
2. **Visual comparison**: Easier to spot differences in data
3. **Publication-ready**: Match journal/paper formatting requirements
4. **Performance**: Potentially faster rendering with shorter strings
5. **User control**: Power users can still see full precision when needed

## Implementation Details

### Files to Modify

1. **ViewProperties.java**
- Add `DATA_DISPLAY_PRECISION` constant
- Add getter/setter methods

2. **UserOptionsHDFPage.java**
- Add precision spinner control
- Load/save precision setting

3. **DataDisplayConverterFactory.java**
- Apply precision when formatting float/double values
- Respect scientific notation setting

4. **Complex Number Formatting**
- Apply to both real and imaginary parts
- Format: `1.23+4.56i` with configurable precision

### Default Behavior

- **Default precision**: 6 decimal places (matches scientific convention)
- **Preserve full precision**: Still store/save full double precision
- **Scientific notation**: Respect existing show/hide setting
- **Complex numbers**: Apply to both real and imaginary parts

### Edge Cases

1. **Very small/large numbers**: Use scientific notation automatically when appropriate
2. **Integer values**: Display without decimal point (1 not 1.000000)
3. **Special values**: NaN, Inf, -Inf display as-is
4. **Complex zero**: 0.0+0.0i simplified appropriately

## Testing Considerations

1. Test with various precision settings (1, 3, 6, 12, 16)
2. Test with values that have floating-point representation issues (0.1, 0.2, 2.1, etc.)
3. Test with complex numbers
4. Test with arrays of different datatypes (float32, float64, float16)
5. Verify setting persists across HDFView sessions

## Related Issues

- Floating-point precision display has been a common user question
- Affects all numeric datatypes: float16, bfloat16, float32, float64, long double
- Particularly important for complex datatypes where precision shows in both components

## Priority

**Medium** - Nice-to-have improvement for user experience, not critical functionality

## Estimated Effort

- Small to Medium (few hours)
- Low complexity - straightforward formatting change
- Well-defined scope

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.