stackabletech / stackabletech/opa-operator

Research: Do we need resource limits for the bundle builder container if we use config maps/pvcs?

Open
#348 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
21
Forks
5
Avg merge
12h 44m
Merged PRs (30d)
11

Description

The PR https://github.com/stackabletech/opa-operator/pull/347 implements resource limits and requests for the opa container. The second container opa-bundle-builder currently does not have any limits.

The opa-bundle-builder basically reads all provided OPA rego config maps and puts the content into a tar bundle. From the OPA docs this could grow to a quite a size.

Currently, the size of the data in ConfigMaps cannot exceed 1MB (etcd limit).

This can become a problem if there are many configmaps with rules / data.
The OpaBundleBuilder uses the tar crate, where it is stated that not all of the content must be explicitly in memory.

If rules are adapted dynamically, this also could put quite a load on the CPU when repacking the bundle.tar.gz (not sure we ever reach that many/big rules though).

This is done when:

  • The tar crate is tested for memory consumption
  • The tar crate is tested for CPU utilization
  • We have some tests / estimates if we need more resources than the default ones for the opa-bundle-builder (e.g. with 1000 - 10000 Configmaps a ~1MB)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing PR #347 and the OpaBundleBuilder usage of the tar crate, then consult the linked OPA resource-utilization documentation. Measure memory and CPU behavior for bundle creation and assess workloads involving roughly 1,000–10,000 ConfigMaps of about 1 MB each. Done means the checklist has measurements or estimates and a clear conclusion about required resource limits.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, rust
Domain
infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.