hydradatabase / hydradatabase/columnar

[Feature]: Please help support new feature to identify and fix the corrupted stripe in large hydra table

Open
#284 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C
Stars
3k
Forks
106
PR merge metrics
No merged PRs in 30d

Description

### What's wrong?

Summary

Currently a decompression/read error on one corrupted stripe aborts reads, VACUUM and index builds for the whole columnar table (DecompressBuffer -> ereport(ERROR)). For very large tables this means a single corrupted stripe can take the whole table offline.

Expected / requested behavior

Please consider an opt-in mode that:

- detects a decompression/read failure for a stripe,

- records the stripe id as “corrupted” in metadata (visible via SQL),

- skips that stripe during reads and maintenance (VACUUM / index build) so the rest of the table continues to work,

- exposes a small admin API/view so operators can list/repair/clear corrupted stripes.

Why this helps

- Makes large tables more resilient: a single corrupted stripe won’t stop queries on the rest of the table, and operators get a clear list of affected stripes to repair.

Minimal implementation idea (optional)

- An opt-in GUC (e.g. columnar.ignore_corrupted_stripes = on) + a reader-side PG_TRY around decompression that logs the stripe and marks it in metadata, returning NULLs for that stripe’s rows.

Thanks
Thanks for the great work on this extension — happy to help sketch or prototype a minimal patch if useful.

Frank

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the reader path around DecompressBuffer -> ereport(ERROR), then follow how VACUUM and index builds process stripes. Done means an opt-in mode can record corrupted stripe IDs in SQL-visible metadata, skip those stripes during reads and maintenance, and provide admin operations to list, repair, and clear them.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, postgresql
Domain
databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.