Extend ArgoCD GitOps demo with DataOps

Stackable data platforms are defined declaratively, enabling automated, standardized, and efficient deployments.

This demo showcases an exemplary GitOps setup for the Stackable data platform using e.g. ArgoCD and distinct Dev/Prod stages. This approach offers consistency, version control, speed, enhanced collaboration, robust security, and observability, making it easier for teams to manage infrastructure as code and adopt Stackable.

For new, non-advanced users, this setup simplifies the learning curve, reduces manual errors, and provides a clear pathway to modern DevOps/GitOps practices.

Prerequisites

• no Dev/Prod stages (only 1 cluster)
• without terraform
• including secrets via git (sealed secrets)

Tasks - DataOps

• Integrate existing demo "trino-taxi-data" in ArgoCD demo
• Add a new airflow DAG to submit Trino jobs (create schema, create table,...)
• PLACEHOLDER: Show progress and refine next step
• Deploy new artifacts via dataops from source to target
• Documentation

Start by reading the existing ArgoCD demo and the “trino-taxi-data” demo, then identify how the demo handles its cluster and sealed secrets. Define the DataOps scope around integrating the demo, adding the Airflow DAG for Trino jobs, deploying artifacts from source to target, and documenting the completed workflow.