[Initiative]: Cloud Native and OCI Compliant Inner-Loop Tooling & Packaging for AI Engineers

### Name

Cloud Native and OCI Compliant Inner-Loop Tooling & Packaging for AI Engineers

### Short description

Integrating the AI developer inner loop into an end-to-end CI/CD process leveraging cloud-native technologies and tooling

### Responsible group

TOC

### Does the initiative belong to a subproject?

Yes

### Subproject name

TOC Artificial Intelligence Initiatives

### Primary contact

Vincent Caldeira (vincent.caldeira@gmail.com)
Savitha Raghunathan (sraghuna@redhat.com)

### Additional contacts

Ricardo Aravena (raravena80@gmail.com)

### Initiative description

**Scope definition**

Focus on the developer inner loop, everything an AI engineer does on a laptop/desktop before code or models ever reach CI/CD in a cloud-native environment:

- Local container workspaces: Reference inner loop workflow using desktop tooling such as Podman Desktop / Podman AI Lab for root-less, GPU-aware experimentation, including template images for PyTorch/LLM stacks and volume-mounted datasets. ​
- Unified model build & run CLI: Hardening inference on developer machine and agentic frameworks to leverage container-based tooling so engineers can easily spin-up inference, RAG and multi-agent services locally with one command.
- Standard packaging of artefacts: Drive convergence between various implementations such as ModelKit, ModelCar towards the emerging ModelPack spec to create a single OCI-manifest that can hold model weights, metadata and SBOM.
- Inner-loop supply-chain security: Integrate Notary v2 / model authenticity and transparency via Sigstore, LF AI & Data Model Openness Framework-generated model & data cards, plus SBOM annotations directly into the OCI artefact so that security & openness are “baked in” before CI. ​
- Fast hand-off to outer loop: Provide reference GitOps flows (Flux/Argo) that pull the signed artefact into KServe with ModelPack image-mount optimisation, and register versions in Kubeflow Model Registry.

**Why it matters for the CNCF**

- Closes the skills gap: Today AI engineers live in Python notebooks while cloud-native tools live in YAML. A container-native inner loop brings AI creators into the CNCF ecosystem early, making Kubernetes the default target platform.
- Eliminates fragmentation: Multiple, incompatible model-packaging attempts (Docker model-CLI, KitOps, ONNX zip files, etc.) slow adoption. A CNCF-backed, OCI-compatible spec creates a neutral home and clear interoperability story. ​
- Raises baseline security & transparency: By embedding MOF openness requirements and Sigstore signing before code hits CI, the sub-stream aligns with industry compliance trends and improves trust across end-to-end supply chains. ​
- Accelerates project reuse: The work provides reusable libraries, CRDs and GitOps templates that every other CNCF AI project (KServe, Kubeflow, TrustyAI, etc.) can import rather than reinventing developer tooling.

**Key technologies & projects involved**

- Container tooling: Podman Desktop, Podman AI Lab, Docker model-runner (observer role)
- Packaging & spec: ModelKit (KitOps), ModelPack, ModelCar, OCI image/artefact spec, Notary v2, Sigstore
- Model runtime & APIs: Ramalama (potential contribution), Agentic Orchestration Frameworks, MCP Servers for tool orchestration
- Kubernetes services: KServe + ModelCars, Kubeflow Model Registry
- Governance & openness: LF AI & DATA Model Openness Framework (MOF) generators, SBOM annotations
- GitOps & automation: Flux, Argo Workflows/Pipelines

### Deliverable(s) or exit criteria

1. An technical POC showing <10 min “idea-to-inference” path for cloud-native agent development on a developer laptop.
2. Clearly documented standards for OCI artefact standardization across runtimes and registries.
3. Specification / procedure to achieve MOF Class III compliant model distributions via any OCI registry.
4. Standardised process for leveraging model signing with artefacts-level provenance to support a verified end-to-end CI/CD reference pipeline including outer loop for AI engineering.