apache / apache/hugegraph-toolchain

[Roadmap] Incremental Rust modernization for HugeGraph Toolchain

Open
#748 0 comments 1 reaction 0 assignees View on GitHub
client enhancement help wanted loader tools
Dominant language
Java
Stars
121
Forks
125
Avg merge
10d 10h
Merged PRs (30d)
3

Description

## Goal

Explore an **incremental Rust implementation** of selected HugeGraph Toolchain components. The intent is to improve safety, portability, startup/runtime efficiency, and contributor experience where Rust provides measurable value.

This is a parent tracking issue, not a commitment to replace the existing implementation wholesale. Existing Java/Go components remain supported while Rust alternatives are designed, benchmarked, and adopted step by step.

## Roadmap at a glance

```text
┌─ Toolchain: first phase ───────────────────────────┐
│ Rust client ─┐ │
│ ├─> shared tests/contracts ─> Tools │
│ Rust Loader ─┘ │
└────────────────────────────────────────────────────┘

v
Server core -> HStore -> PD -> Computer -> Vermeer/OLAP

Docs, compatibility checks, and benchmarks span every phase.
```

## Proposed priority

1. **Rust client and Loader (parallel)**
- Use the Java client and current Loader behavior as the compatibility reference.
- Start with a minimal client: transport, authentication, schema, graph, and Gremlin APIs.
- Build a Loader pipeline on top of the client with bounded memory, backpressure, retry, and resumability.
2. **Tools**
- Port small, well-bounded commands after the client APIs stabilize.
- Preserve command-line behavior and backup/restore compatibility.
3. **Other components**
- Evaluate independently; migrate only when the benefit and maintenance cost are clear.

## Delivery model

```text
proposal/RFC -> minimal vertical slice -> compatibility tests
-> benchmark -> opt-in preview -> production-ready candidate
```

## Planning guardrails (required)

Before implementation starts, each child issue **must** define:

- scope, owner/mentor, and non-goals;
- API/CLI, protocol, data/storage-format, and operational compatibility;
- differential, cross-language parity, concurrency, fault-injection, and upgrade/rollback tests, where applicable;
- reproducible latency, throughput, memory, startup, and recovery benchmarks;
- security/dependency checks (`rustfmt`, `clippy`, audit) and supported platforms;
- observability, packaging/deployment, coexistence, migration, and rollback plans;
- explicit acceptance criteria and user/operator/contributor documentation.

No large rewrite PR should start before its design and acceptance criteria are reviewed. Correctness, recoverability, and compatibility take precedence over headline benchmark numbers. Small, reviewable PRs and stable interoperability boundaries are required; an unplanned rewrite PR may be closed or asked to return to the RFC/design stage.

## Candidate work items

- [ ] Agree on the Rust workspace/repository layout and minimum supported Rust version
- [ ] Define the compatibility matrix and shared test fixtures
- [ ] Create the minimal Rust client design issue
- [ ] Create the Loader architecture/design issue
- [ ] Establish CI, lint, dependency, packaging, and release conventions
- [ ] Publish baseline benchmarks against the current implementation
- [ ] Split approved work into newcomer-friendly child issues

This checklist will link to concrete child issues as proposals are accepted.

## Project-wide roadmap

- Toolchain (this issue): apache/hugegraph-toolchain#748
- Server / HStore / PD: apache/hugegraph#3110
- Computer / Vermeer: apache/hugegraph-computer#355
- Documentation hub: apache/hugegraph-doc#462

## How to participate

Interested in **open source, Rust, graph systems, databases, or infrastructure**? Comment with the area you want to work on and any relevant experience. RFC review, benchmarks, tests, documentation, and small implementation tasks are all valuable.

First-time contributors can start with the [HugeGraph contribution guide](https://hugegraph.apache.org/docs/contribution-guidelines/contribute/) and the repository's `good first issue` / `help wanted` tasks.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the roadmap checklist and the HugeGraph contribution guide linked in the issue. There are no files, tests, or entry points identified; this parent issue is complete when the workspace, compatibility, CI, benchmark, and design work is split into reviewed child issues with explicit acceptance criteria.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, java, rust
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.