rucio / rucio/k8s-tutorial

Add multi-storage support with managed upload workflow

Open
#49 1 comment 0 reactions 1 assignee View on GitHub

@mgajek-cern is already working on this.

Since Feb 16, 2026.

discussion enhancement
Dominant language
Shell
Stars
12
Forks
30
PR merge metrics
No merged PRs in 30d

Description

Summary

Extend k8s-tutorial to demonstrate Rucio data management across multiple storage technologies (S3, dCache) using the managed upload pattern.

Motivation

Current tutorial only demonstrates XRootD storage. Production Rucio deployments typically integrate with diverse storage backends (object stores, grid storage, distributed filesystems).

Proposed Changes
  • Add S3-like storage, e.g. Localstack (2 instances) - cloud-native object storage
  • Add dCache storage (2 instances) - grid storage with WebDAV
  • Update workflow script demonstrating managed upload pattern
  • Update deployment scripts to handle all storage types preserving previous behavior and opt-in feature flags for multi-storage demonstration

Workflow demonstrated:

  1. Upload files to source RSEs (XRD1, S3_1, DCACHE_1)
  2. Create datasets and containers
  3. Create replication rules to destination RSEs
  4. FTS3 orchestrates third-party copy transfers
  5. Monitor transfer completion
Testing
./scripts/deploy-rucio.sh --enable-all-storage

# Auto-detects and configures enabled storages
./scripts/use-rucio.sh
Future Work

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.