apache / apache/opendal

Tracking issue: Split ecosystem integrations out of the core repository

Open
#7,584 0 comments 2 reactions 0 assignees View on GitHub
enhancement integrations/object_store
Dominant language
Rust
Stars
5.4k
Forks
825
Avg merge
1d 14m
Merged PRs (30d)
127

Description

## Feature Description

OpenDAL currently keeps several ecosystem integrations under `integrations/`, including Rust adapters such as `object_store_opendal`, `parquet_opendal`, `dav-server-opendalfs`, `unftp-sbe-opendal`, and the Spring integration modules.

These packages are ecosystem adapters rather than part of OpenDAL's core storage abstraction. We should split them out of the core repository so each integration can follow its own ecosystem, dependency, CI, and release cadence while keeping OpenDAL core focused on the abstraction, services, layers, and testkit.

## Problem and Solution

Keeping integrations in the core repository creates several forms of coupling:

- core CI and release workflows need to know about integration-specific build systems and package versions;
- integration versions can be mistaken as lockstep with `opendal`, even when they should be package-specific;
- ecosystem adapters need to follow upstream dependency changes that are unrelated to core development;
- release and documentation workflows become harder to reason about as bindings, core crates, and integrations are mixed together.

The proposed direction is to split ecosystem integrations into separate repository/repositories, while keeping explicit compatibility contracts and documentation discoverability from the OpenDAL website.

## Proposed Migration Order

- [ ] Decide the target repository layout: one `opendal-integrations` repository or separate repositories per integration family.
- [ ] Define ownership and release policy for split integrations.
- [ ] Define versioning policy: integration packages use independent semver and must not imply lockstep with `opendal` unless explicitly documented.
- [ ] Move Spring integration first, since it has an independent Maven/Spring build and release model.
- [ ] Move `dav-server-opendalfs`, `unftp-sbe-opendal`, and `parquet_opendal` after the repository and release workflow are ready.
- [ ] Move `object_store_opendal` with extra care because it is a high-impact bridge into the Rust data ecosystem.
- [ ] Keep downstream/compatibility validation for important adapters, especially `object_store_opendal`, when core APIs change.
- [ ] Update OpenDAL website docs so users can still discover supported integrations from the main project site.
- [ ] Update release tooling so core releases no longer publish split integration packages.
- [ ] Update CI, Dependabot, docs generation, and behavior-test workflows that currently reference `integrations/**` paths.
- [ ] Preserve issue/PR/history references or provide clear migration links from the old paths.

## Non-Goals

- This issue does not propose changing OpenDAL's core public API.
- This issue does not propose moving services, layers, testkit, or behavior tests out of the core repository.
- This issue does not require all integrations to share one release cadence.
- This issue does not require `object_store_opendal` to remain version-aligned with `opendal`.

## Acceptance Criteria

- Integration packages are maintained outside the core repository with clear repository ownership.
- Each split integration has documented compatibility expectations with supported `opendal` versions.
- OpenDAL core release workflows no longer publish integration packages directly.
- Core CI keeps only the compatibility checks that are intentionally required for core changes.
- Users can find integration documentation from the OpenDAL website after the split.

Contributor guide

Open the contributing guide

Research direction

This is a repository-level tracking issue rather than a named code change. Start by tracing integrations/** through CI, release tooling, Dependabot, docs generation, behavior-test workflows, and website documentation; first resolve the target repository and ownership and versioning decisions. Done means integrations are maintained outside core, compatibility is documented, core releases no longer publish them, required checks remain, and users can find the documentation.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, spring
Domain
build-system, ci-cd, documentation, release
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.