Flagsmith / Flagsmith/flagsmith
Migrate to native flag-engine v10 APIs and remove vendored models
- Dominant language
- Python
- Stars
- 6.6k
- Forks
- 567
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 121
Description
Phase 1 of the flag-engine v10 migration (#5991, PR #6653/#6741) upgraded
`flagsmith-flag-engine` to `^10.0.3` and vendored Pydantic models from the
`fix/missing-export` branch to maintain backwards compatibility.
This issue tracks **Phase 2**: fully migrating to flag-engine v10's native APIs
and removing the vendored code.
## Goals
### `util.mappers.engine` — Django models → `EvaluationContext`
Engine mappers should map from Django ORM models to flag-engine's
`EvaluationContext`. Core API should define its own `SegmentMetadata` and
`FeatureMetadata` TypedDicts (e.g. in a `types.py` module) and use them as type
arguments for the generic `EvaluationContext[SegmentMetadata, FeatureMetadata]`.
### `util.mappers.dynamodb` — Django models → `flagsmith_schemas.dynamodb` TypedDicts
DynamoDB mappers should map from Django ORM models directly to the TypedDicts
defined in `flagsmith_schemas.dynamodb` (e.g. `Environment`,
`EnvironmentV2IdentityOverride`, `Identity`), eliminating the intermediate
Pydantic model layer.
### `util.mappers.sdk` — Django models → `flagsmith_schemas.api` TypedDicts
SDK mappers should map from Django ORM models directly to the TypedDicts defined
in `flagsmith_schemas.api` (e.g. `V1EnvironmentDocumentResponse`), eliminating
the intermediate Pydantic model layer.
### Typing discipline
No new `# type: ignore` comments should be introduced. `typing.cast` calls
should be avoided — types should be correct by construction.
## Tasks
- [ ] Define Core API `SegmentMetadata` and `FeatureMetadata` TypedDicts for use
as `EvaluationContext` type arguments.
- [ ] Rewrite `util.mappers.engine` to map Django models directly to
`EvaluationContext[SegmentMetadata, FeatureMetadata]`, removing all
vendored Pydantic model usage.
- [x] Migrate segment evaluation from vendored `is_context_in_segment` to
flag-engine's `get_evaluation_result` (#6669, PR #6896).
- [ ] Use `get_evaluation_result` for full identity flag evaluation, replacing
`Identity.get_all_feature_states()` + serialiser-time multivariate
resolution (see [consumers table](#consumers-of-identity-based-feature-state-resolution) below).
- [ ] Add `map_feature_state_to_feature_context` mapper in
`util/mappers/engine.py` to convert Django `FeatureState` (with
prefetched MV values) into a `FeatureContext` TypedDict with `variants`.
- [ ] Populate `EvaluationContext.features` alongside segments in
`Identity.get_all_feature_states()` (or a replacement method), so
`get_evaluation_result` resolves segments, applies segment overrides,
and resolves multivariate values in a single pass.
- [ ] Adapt SDK serialisers to consume `FlagResult` dicts from the engine
instead of calling `get_feature_state_value_by_hash_key()` /
`get_feature_state_value(identity=...)`.
- [ ] Update integration callers (amplitude, heap, mixpanel, rudderstack,
segment) to receive already-resolved values from the evaluation result
rather than calling `get_feature_state_value(identity=identity)`.
- [ ] Remove `FeatureState.get_multivariate_feature_state_value()` and
`FeatureState.get_feature_state_value_by_hash_key()` (or simplify for
non-identity cases).
- [ ] Remove `get_hashed_percentage_for_object_ids` from
`util/engine_models/utils/hashing.py` and
`util/engine_models/features/models.py`.
- [ ] Rewrite `util.mappers.dynamodb` to map Django models directly to
`flagsmith_schemas.dynamodb` TypedDicts, removing Pydantic `BaseModel`
serialisation and `FeatureStateModel.parse_obj()` calls.
- [ ] Rewrite `util.mappers.sdk` to map Django models directly to
`flagsmith_schemas.api` TypedDicts, removing vendored model usage.
- [ ] Replace `IdentityOverrideV2` in `environments/dynamodb/types.py` with
`flagsmith_schemas.dynamodb.EnvironmentV2IdentityOverride` or an
equivalent TypedDict that does not depend on vendored models.
- [ ] Update remaining consumers of `util.engine_models` (see table below).
- [ ] Remove `api/util/engine_models/` entirely.
- [ ] Remove `pydantic-collections` from `pyproject.toml`.
- [ ] Ensure no new `# type: ignore` comments or `typing.cast` calls are
introduced; all types should be correct by construction.
- [ ] Ensure all tests pass and diff coverage remains at 100%.
### Consumers of identity-based feature state resolution
Code that resolves feature state values via the identity hash key /
multivariate resolution path. These need migrating to `get_evaluation_result`.
| File | Method / usage | What it does |
|------|---------------|--------------|
| `features/models.py` | `FeatureState.get_feature_state_value(identity=)` | Public entry point — extracts hash key, delegates to `get_feature_state_value_by_hash_key` |
| `features/models.py` | `FeatureState.get_feature_state_value_by_hash_key()` | Calls `get_multivariate_feature_state_value()` for MV features |
| `features/models.py` | `FeatureState.get_multivariate_feature_state_value()` | Hashes `[feature_state.id, identity_hash_key]` to pick MV variant |
| `features/serializers.py` | `FeatureStateSerializerFull.get_feature_state_value()` | Calls `obj.get_feature_state_value(identity=...)` |
| `features/serializers.py` | `SDKFeatureStateSerializer.get_feature_state_value()` | Calls `obj.get_feature_state_value(identity=...)` |
| `features/serializers.py` | `SDKFeatureStatesQuerySerializer` context | Calls `obj.get_feature_state_value(identity=...)` |
| `environments/identities/serializers.py` | `IdentityAllFeatureStatesSerializer.get_feature_state_value()` | Calls `instance.get_feature_state_value_by_hash_key(hash_key)` or `instance.get_value(hash_key)` |
| `edge_api/identities/serializers.py` | `FeatureStateValueEdgeIdentityField.to_representation()` | Calls `obj.get_value(identity_id=...)` on Pydantic FeatureStateModel |
| `integrations/amplitude/amplitude.py` | `AmplitudeWrapper.generate_user_data()` | Calls `feature_state.get_feature_state_value(identity=identity)` |
| `integrations/heap/heap.py` | `HeapWrapper.generate_user_data()` | Calls `feature_state.get_feature_state_value(identity=identity)` |
| `integrations/mixpanel/mixpanel.py` | `MixpanelWrapper.generate_user_data()` | Calls `feature_state.get_feature_state_value(identity=identity)` |
| `integrations/rudderstack/rudderstack.py` | `RudderstackWrapper.generate_user_data()` | Calls `feature_state.get_feature_state_value(identity=identity)` |
| `integrations/segment/segment.py` | `SegmentWrapper.generate_user_data()` | Calls `feature_state.get_feature_state_value(identity=identity)` |
### Consumers of vendored segment evaluation (solved in #6669)
Code that still uses the vendored `is_context_in_segment` / `get_context_segments` wrappers.
| File | Import / usage | Proposed change |
|------|---------------|-----------------|
| `util/engine_models/context/mappers.py` | `is_context_in_segment()`, `get_context_segments()` | Remove — callers should use `get_evaluation_result` directly |
| `environments/dynamodb/wrappers/identity_wrapper.py` | `get_context_segments()`, `map_environment_identity_to_context()` | Use `EvaluationContext` + `get_evaluation_result` for segment matching |
### Consumers of `util.engine_models` (non-test files)
| File | Imports | Proposed change |
|------|---------|-----------------|
| `edge_api/identities/models.py` | `IdentityModel`, `IdentityFeaturesList`, `FeatureStateModel` | Use `flagsmith_schemas.dynamodb.Identity` / `FeatureState` TypedDicts as storage representation |
| `edge_api/identities/serializers.py` | `FeatureModel`, `FeatureStateModel`, `MultivariateFeatureOptionModel`, `MultivariateFeatureStateValueModel`, `IdentityModel`, `DuplicateFeatureState` | Construct `flagsmith_schemas.dynamodb` TypedDicts directly from request data |
| `edge_api/identities/views.py` | `IdentityModel`, `IdentityFeaturesList`, `TraitModel` | Use `flagsmith_schemas.dynamodb.Identity` / `Trait` TypedDicts |
| `edge_api/identities/utils.py` | `FeatureStateModel` | Use `flagsmith_schemas.dynamodb.FeatureState` for change tracking |
| `edge_api/identities/export.py` | `map_any_value_to_trait_value` | Use `flag_engine.ContextValue` or a standalone coercion function |
| `environments/dynamodb/wrappers/identity_wrapper.py` | `EnvironmentModel`, `IdentityModel`, `map_environment_identity_to_context`, `get_context_segments` | Use `EvaluationContext` + `get_evaluation_result` for segment matching |
| `environments/dynamodb/services.py` | `IdentityModel` | Use `TypeAdapter[flagsmith_schemas.dynamodb.Identity]` for validation |
| `environments/dynamodb/types.py` | `FeatureStateModel` | Replace `IdentityOverrideV2` with `flagsmith_schemas.dynamodb.EnvironmentV2IdentityOverride` |
| `environments/identities/serializers.py` | `FeatureStateModel` | Use `flagsmith_schemas.api.FeatureState` in `get_feature_state_value()` |
| `organisations/models.py` | `TraitValue` | Use `flag_engine.ContextValue` directly |
| `app/pagination.py` | `IdentityModel` | Use `TypeAdapter[flagsmith_schemas.dynamodb.Identity]` for validation |
| `util/mappers/engine.py` | All models (15+ types) | Produce `EvaluationContext[SegmentMetadata, FeatureMetadata]` directly |
| `util/mappers/dynamodb.py` | `FeatureStateModel` | Produce `flagsmith_schemas.dynamodb` TypedDicts directly; drop `parse_obj()` |
## Related
- Phase 1: #5991 (closed), PR #6653, PR #6741
- Sub-task: #6669
- Edge API counterpart: Flagsmith/edge-api#584 (transferred from #6571)
- Common package types: https://github.com/Flagsmith/flagsmith-common/issues/128 (closed)
Contributor guide
Assessment
This issue has not been assessed yet.