crossplane-contrib / crossplane-contrib/provider-sql
Perf issues when provisioning databases from a template
- Dominant language
- Go
- Stars
- 154
- Forks
- 119
- Avg merge
- 6d 17h
- Merged PRs (30d)
- 8
Description
There's several things that currently make database provisioning from a template sub-optimal:
1. hardcoded reconciler timeout limit of [1 minute](https://github.com/crossplane/crossplane-runtime/blob/main/pkg/reconciler/managed/reconciler.go#L49C2-L49C18) - which is not enough to provision DBs from large many GB templates; current provider should expose a way to override that value,
2. default "from template" strategy, which is sub-optimal for large databases (hopefully will be addressed when https://github.com/crossplane-contrib/provider-sql/issues/341 is closed),
3. inability to serialize all database provision calls - when many DBs are created from template at once, database instance's disk gets saturated and all attempts fail due to item 1 above,
My use case is this: provision database from a template, for what we call "review apps", e.g. apps created from a PR/branch for review/demo/testing reasons.
Problem is especially visible, when attempting to create multiple DBs for multiple "review apps" that must exist at once (in scenarios like "1 app per e2e suite").
Happy to open PRs for issues 1 and 3 if there's interest. PR for issue 2 already exists as far as I can tell.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with the linked crossplane-runtime pkg/reconciler/managed/reconciler.go timeout definition and inspect provider-sql's database provisioning and reconciliation entry points. Separate the requested timeout override and serialized provisioning work from the template-strategy change tracked in issue 341. Done means the first two concerns have an agreed implementation and coverage for large or concurrent template-based database creation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, sql
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100