Flagsmith / Flagsmith/flagsmith

A v2 versioning write costs queries proportional to the flag's segment overrides

Open
#8,312 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
6.6k
Forks
567
Avg merge
1d 13h
Merged PRs (30d)
121

Description

Every write to a flag in a v2-versioned environment opens a new `EnvironmentFeatureVersion` and publishes it. Both halves of that cost grow with the number of feature states the flag has, so a flag with many segment overrides pays a query count proportional to its overrides.

Measured on `main` with `CaptureQueriesContext` around `EnvironmentFeatureVersion.objects.create(...)` and `version.publish(...)`, for a flag with 1 and with 3 segment overrides (2 and 4 feature states):

| Step | 1 override | 3 overrides | Per extra state |
| ------------------------------------------ | ---------- | ----------- | --------------- |
| `EnvironmentFeatureVersion.objects.create` | 26 | 54 | ~8 |
| `version.publish` | 45 | 61 | ~5 |

Extrapolated, a flag with 50 segment overrides costs roughly 400 queries to open a version and 250 to publish it. Every v2 write path pays this: the versioning serializers, change request commits, and the experimental update-flag endpoints alike.

### Creating a version copies row by row

`FeatureState.clone` (`api/features/models.py:661`) is a `deepcopy` plus `save()` per state, and `FeatureSegment.clone` (`api/features/models.py:328`) does the same. Per cloned state the create issues, roughly:

- `SELECT features_feature`
- `SELECT features_featurestatevalue`
- `INSERT features_featurestate` and `INSERT features_historicalfeaturestate`
- `INSERT features_featurestatevalue` and `INSERT features_historicalfeaturestatevalue`
- a `SAVEPOINT` / `RELEASE` pair

Some copying is inherent to immutable versions, but the row-at-a-time shape is not: `bulk_create` for the states and their values, with the history rows batched alongside, would collapse most of it.

### Publishing re-fetches relations per state

This half looks like a plain N+1 rather than inherent work. Between 1 and 3 overrides, `publish` goes from 45 to 61 queries, and the growth is all in per-state relation lookups:

| Query | 1 override | 3 overrides |
| --------------------------------------------------- | ---------- | ----------- |
| `SELECT multivariate_multivariatefeaturestatevalue` | 6 | 10 |
| `SELECT features_feature` | 4 | 8 |
| `SELECT features_featurestatevalue` | 4 | 8 |
| `SELECT features_featuresegment` | 3 | 7 |

Whatever the publish path and its signal handlers iterate should be able to hydrate these with `select_related` / `prefetch_related` in one pass.

### Suggested scope

1. Flatten the publish-side N+1 first — it is the cheaper fix and is not intrinsic to versioning.
2. Then consider bulk-creating the cloned states, values and multivariate values when a version is created.

Worth pinning either fix with a query-count test at the service level.

Contributor guide

Open the contributing guide

Research direction

Start with FeatureState.clone and FeatureSegment.clone in api/features/models.py, then trace EnvironmentFeatureVersion.objects.create(...) and version.publish(...) using the reported CaptureQueriesContext measurements. Inspect the publish path and its signal handlers for per-state relation lookups, and add a service-level query-count test covering the selected optimization scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.