apache / apache/celix

[Proposal] Bundle Store: Adopting OCI Registry as Storage for Apache Celix

Open
#830 1 comment 1 reaction 0 assignees View on GitHub
discussion needed kind/wish
Dominant language
C
Stars
193
Forks
99
Avg merge
1m
Merged PRs (30d)
1

Description

### Description

ASF Celix bundles are currently distributed and loaded as ZIP files, which requires a runtime dependency on `libzip`. This proposal details the concept of a **Bundle Store** architecture designed to decouple the distribution format from runtime execution. By adopting a Content-Addressable Storage (CAS) model (aligned with the **OCI Registry as Storage (ORAS)** concept) bundles can be provisioned via ZIP, TAR, or OCI layers, while moving the bundle runtime usage to a directory-based system.

### Rationale

* **Removal of Runtime Overhead:** Transitioning to a directory-based store eliminates the need for `libzip` and runtime decompression.
* **OCI Ecosystem Alignment:** Leveraging the ORAS concept for bundle distribution and possible signing.
* **Security & Reliability:** Explicitly separates immutable bundle code from application state.

### Storage Management: `celix-bnd-storage`

The management of the bundle store and the association of storage tiers is handled by a new utility: **`celix-bnd-storage`**.

#### Configuration (`~/.config/celix/bnd-storage.json`)

This human-editable file defines the global environment settings:

* **Storage Roots:** Base paths for the **Bundle Store**, **Persistent Storage**, and **Volatile Storage**.
* **Policy Settings:** Default garbage collection (GC) rules and retention periods.

#### Instance (Bundle) State (`/var/run/celix/instances.d/` or `~/.local/share/celix/state/instances.d/`)

To ensure concurrency safety and remove the usage of a state directory in the CWD, a directory-based **Instance State** is introduced. Each active framework instance maintains its own JSON entry named after its SHA-256 ID. This registry tracks:

* **Instance Mapping:** Linkage between the **Framework Instance SHA-256** and its assigned **Friendly Name**.
* **Process Tracking:** The current **Process ID (PID)** and start time.

### Framework Instance Identification

The framework transitions from random UUIDs to deterministic **SHA-256 Framework Instance IDs**.

1. **Identity Derivation:** If an instance ID is not explicitly provided, the Celix launcher generates a SHA-256 hash based on:
* The absolute path of the application executable
* The absolute path of the current working directory (CWD).

2. **Naming Approach:** Instances are assigned "friendly names" (e.g., `focused_hawking`) mapped to the SHA-256 ID within the State Registry entries (nice to have).
3. **Launcher Integration:** The launcher supports the `--instance_id` argument to manually specify a SHA-256 ID. And optionally support a `--instance_name` argument.

### Architecture: The 3-Tier Filesystem Model

The framework interacts with three distinct filesystem areas:

1. **Bundle Store (Immutable):**
* `blobs/sha256/`: Contains unpacked bundle content, indexed by the SHA-256 hash. If possible, these directories should be mounted or marked as **read-only** to ensure integrity.
* `tags//`: A structure of symbolic links, linking to unpacked bundle blob entries.

2. **Volatile Storage:**
* Located at `//`. Intended for `tmpfs` storage.

3. **Persistent Storage:**
* Located at `//`. Reserved for long-term state that survives restarts.

### Proposed API Changes

The `celix_bundle.h` and `celix_framework.h` interfaces are extended:

* **Immutable Resources:**
* `const char* celix_bundle_getBundleStorePath(const celix_bundle_t* bnd, const char* path)`: Returns the absolute path to a file within the **Bundle Store** for the specific bundle.

* **Instance Storage:**
* `char* celix_bundle_getPersistentStoragePath(const celix_bundle_t* bnd)`: Returns the path to the persistent directory dedicated to the bundle for the specific framework instance.
* `char* celix_bundle_getVolatileStoragePath(const celix_bundle_t* bnd)`: Returns the path to the volatile directory for the current run.

* **Framework**
* `const char* celix_framework_getInstanceId(const celix_framework_t* fw)`: Returns the framework instance id (SHA256).
* `const char* celix_framework_getName(const celix_framework_t* fw)`: Returns the framework (human friendly) name.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.