Flagsmith / Flagsmith/flagsmith
Segment percentage split evaluation is not consistent between Core API and local evaluation
- Dominant language
- Python
- Stars
- 6.6k
- Forks
- 567
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 121
Description
**Describe the bug**
Identities are evaluated differently for segments containing a percentage split rule when evaluated against the Core API versus local evaluation mode in SDKs.
**To Reproduce**
Steps to reproduce the behavior:
1. Run the Core API (e.g. self hosted)
2. Create a segment with a percentage split rule of 50% and add a segment override to a basic feature in a given environment
3. Instantiate two instances of a client (e.g. Node.js), one running in local evaluation mode and one in remote
4. Evaluate the flag for multiple identities in both instances of the client and compare the results
5. Notice that the state of the flag can differ for certain identities between local and remote evaluation
**Expected behavior**
The result is the same for both local and remote evaluation for all identities
**Screenshots**
N/a
**How are you running Flagsmith?**
- [x] Self Hosted with Docker
- [x] Self Hosted with Kubernetes
- [x] SaaS at flagsmith.com (only for Core projects)
- [ ] Some other way
**Additional context**
This will affect the following clients when running in local evaluation mode:
- [x] PHP (Matt)
- [x] Node.js (Matt)
- [x] Java (Kim)
- [x] Elixir (Matt)
- [x] .NET (Kim)
- [x] Go (Kim)
Note that python, ruby, rust are not affected as they will use the django id if it exists.
---
# 2026 updates
TODO: Update above text to match present time.
## Acceptance criteria
Extracted from https://github.com/Flagsmith/flagsmith/issues/6317.
- [ ] Remove `identity.key` field from `sdk/evaluation-context.json` schema
- [ ] Update engine-test-data to remove explicit `identity.key` from test cases
- [ ] Update documentation referencing `identity.key` as user-settable field — it should be internal
Contributor guide
Assessment
This issue has not been assessed yet.