apache / apache/incubator-graphar
[FEATURE][Java] Add a GraphAr dataset validator
- Dominant language
- C++
- Stars
- 373
- Forks
- 93
- Avg merge
- 7d 21h
- Merged PRs (30d)
- 7
Description
### Describe the enhancement requested
Add a dataset validator that checks a published GraphAr dataset against the format rules instead of failing later inside a reader. Given a graph URI it reports concrete, located issues: metadata that does not describe the files on storage, chunk counts that disagree with the declared chunk size, offset chunks that are not monotonic, and adjacency chunks that reference vertices outside the declared count.
Component(s): Java
## Scope
- Add `graphar-validator` with a `DatasetValidator` that walks metadata and storage and returns a `ValidationReport` of located `ValidationIssue` values.
- Depend only on `graphar-info`, `graphar-core`, `graphar-io-api` and `graphar-storage-api`, so validation stays independent of any one format or backend.
- Add tests over deliberately corrupted datasets: missing chunk file, wrong chunk count, non-monotonic offsets, out-of-range vertex id.
## Non-goals
No repair or migration of a broken dataset, no reader facade, no new format backend, and no validation of user payload semantics beyond what the GraphAr format itself states.
## Compatibility and acceptance
A canonical dataset from the `testing` fixtures must validate clean; each corrupted fixture must produce exactly the issue that describes it, with the offending path. Java 11 compatible.
## Dependency order
Its tests build datasets through the write path, so this slice depends on `graphar-core` (#967, PR #968), a merged format backend (#964), and the write facade (#973). It should land last of the four.
Contributor guide
Research direction
Start with the graphar-core, merged format backend, and write facade dependency order, then inspect the testing fixtures and metadata/storage APIs. Implement graphar-validator's DatasetValidator to return located ValidationIssue values in a ValidationReport for the four corrupted cases. Done means the canonical testing fixture validates clean, each corruption yields exactly its described path-specific issue, and Java 11 compatibility is preserved.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data, testing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100