cncf / cncf/toc

[Initiative]: Scale and performance testing guidance

Open
#2,233 4 comments 2 reactions 0 assignees View on GitHub
help wanted init/not-started kind/initiative needs-triage sub/project-reviews tag/operational-resilience
Dominant language
HTML
Stars
1.9k
Forks
724
Avg merge
6d 12h
Merged PRs (30d)
4

Description

### Name

Scale and performance testing guidance for CNCF projects

### Short description

Define performance testing expectations by project type so DD reviewers and projects have clear benchmarks to target.

### Responsible group

TAG Operational Resilience

### Does the initiative belong to a subproject?

No

### Subproject name

### Primary contact

TBD — TAG Operational Resilience lead

### Additional contacts

@angellk (TOC Chair, DD finding source)

### Origin

DD finding (automated)

### Initiative description

**This initiative is a request from the TOC.** During due diligence reviews, the TOC evaluates projects against incubation and graduation criteria. When the same finding appears across multiple projects, it signals an ecosystem-wide gap that would be better addressed through standardized guidance than repeated per-project recommendations.

This finding has appeared in **9 of 42 DD reports** (21%) scanned over the last 5 years, including cncf/toc#2198 (HAMi), cncf/toc#2057 (Kyverno), cncf/toc#2038 (Microcks), cncf/toc#1165 (KubeEdge), and others. The most recent DD to surface this was [Kyverno graduation DD](https://github.com/cncf/toc/pull/2057) (merged 2026-03-17).

DDs frequently recommend performance benchmark data, scale testing, or load testing documentation, but there is no standardized expectation for what constitutes adequate testing by project type. A database operator has different scale testing needs than a CLI tool or a controller. The Kyverno DD specifically called for "more proactive, high-scale performance testing internally" and the HAMi DD recommended "performance benchmark data covering expected overhead and isolation quality across common concurrent workload scenarios."

**Scope:** Define performance testing expectations segmented by project type (database/storage, controller/operator, CLI tool, library, specification). Include guidance on benchmark methodology, reproducibility, what to publish, and how DD reviewers should evaluate performance testing adequacy. Aligns with TAG Operational Resilience charter scope: "Performance" and "Testing."

**Timeline:** TBD

### Deliverable(s) or exit criteria

- [ ] Guidance document: performance and scale testing expectations by project type
- [ ] Example benchmark configurations for common project categories
- [ ] DD checklist item update — PR to graduation application template

### Tracking document for meeting and progress

TBD

Contributor guide

Open the contributing guide

Research direction

Start with the initiative description and review the cited DD reports, including cncf/toc#2198, #2057, #2038, and #1165, to compare the performance-testing findings. Define guidance by the listed project types, covering methodology, reproducibility, published results, and reviewer evaluation. Done means the guidance document, example benchmark configurations, and DD checklist update are completed.

Written by the indexing model from the issue text.

Assessment

Domain
documentation, performance, testing
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.