crossplane-contrib / crossplane-contrib/provider-sql

Perf issues when provisioning databases from a template

Open
#358 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
154
Forks
119
Avg merge
6d 17h
Merged PRs (30d)
8

Description

There's several things that currently make database provisioning from a template sub-optimal:

1. hardcoded reconciler timeout limit of [1 minute](https://github.com/crossplane/crossplane-runtime/blob/main/pkg/reconciler/managed/reconciler.go#L49C2-L49C18) - which is not enough to provision DBs from large many GB templates; current provider should expose a way to override that value,
2. default "from template" strategy, which is sub-optimal for large databases (hopefully will be addressed when https://github.com/crossplane-contrib/provider-sql/issues/341 is closed),
3. inability to serialize all database provision calls - when many DBs are created from template at once, database instance's disk gets saturated and all attempts fail due to item 1 above,

My use case is this: provision database from a template, for what we call "review apps", e.g. apps created from a PR/branch for review/demo/testing reasons.

Problem is especially visible, when attempting to create multiple DBs for multiple "review apps" that must exist at once (in scenarios like "1 app per e2e suite").

Happy to open PRs for issues 1 and 3 if there's interest. PR for issue 2 already exists as far as I can tell.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the linked crossplane-runtime pkg/reconciler/managed/reconciler.go timeout definition and inspect provider-sql's database provisioning and reconciliation entry points. Separate the requested timeout override and serialized provisioning work from the template-strategy change tracked in issue 341. Done means the first two concerns have an agreed implementation and coverage for large or concurrent template-based database creation.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, sql
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.