open-feature / open-feature/spec

The provider contract does not say what determines a flag value's type

Open
#433 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.3k
Forks
58
Avg merge
2d 18h
Merged PRs (30d)
2

Description

The gap

The specification never says what the type of a flag value is, and two separate places depend on the answer.

TYPE_MISMATCH appears exactly once in the whole specification — as a row in the error code table:

| TYPE_MISMATCH | The type of the flag value does not match the expected type. |

No requirement anywhere obliges a provider to raise it. The only normative statement about value type is Requirement 1.3.4, and it is a SHOULD on the client:

The client SHOULD guarantee the returned value of any typed flag evaluation method is of the expected type. If the value returned by the underlying provider implementation does not match the expected type, it's to be considered abnormal execution, and the supplied default value should be returned.

The provider requirements (2.2.1 – 2.2.10) do not mention type at all. Requirement 2.2.3 says only:

In cases of normal execution, the provider MUST populate the resolution details structure's value field with the resolved flag value.

There are zero occurrences of coerce, cast, convert or stringify in the specification.

Why it matters: an untyped backend has no wrong answer

Some flag management systems store every flag value as a string. Flipt does; some Flagsmith configurations do.

For such a provider, a flag whose canonical definition is 10 is stored — and returned — as "10". Ask it through the string accessor and it returns "10" with no error. Is that a type mismatch?

  • The flag's type, as this provider sees it, is string. Requirement 2.2.3 asked for the resolved flag value and it supplied one. Normal execution.
  • The flag's type, as its canonical definition states, is integer. The caller asked for a string. Abnormal execution, TYPE_MISMATCH.

Both readings are consistent with the specification as written, because the specification never picks which notion of "the type of the flag value" is authoritative — the backend's storage type, or the type declared where the flag was defined.

This is not confined to numbers. Every type has a string representation:

flag stored as asked as string honest result?
boolean-flag "true" "true" no mismatch to report
integer-flag "10" "10" no mismatch to report
float-flag "0.5" "0.5" no mismatch to report
object-flag JSON text JSON text no mismatch to report

And the reverse direction is a provider choice rather than a consequence: a provider over that same backend could parse "true" and satisfy the boolean accessor too. Nothing forbids it, so a provider that parses aggressively is as conformant as one that does not — and the two behave observably differently for the same flag and the same call.

Relationship to #430

#430 asks what a provider must do when a numeric value does not fit the accessor it was asked through — is 0.5 an integer, and how wide is an integer accessor. That question presupposes the value is a number.

This issue is upstream of it: what fixes a flag's type in the first place, when the backend does not have types? Answer that, and #430 becomes the numeric special case of it. Leave it unanswered, and #430 can only be answered for backends that happen to be typed.

Closed issue #22 raised "typing of flag values" in 2022 and was closed once the typed-accessor proposal landed. That proposal settled what the accessors are; it did not settle what a value's type is, or who decides.

Why it is worth closing rather than leaving to idiom

The vacuum is already being filled locally, differently, by people who cannot avoid the question.

The provider conformance suite (#417, #423) had four scenarios asserting TYPE_MISMATCH for a non-string flag requested as a string, as mandatory requirements. They were mandatory on the reasoning that "is 0.5 an integer?" has a defensible wrong answer whereas "is a string a boolean?" does not. That reasoning is sound for the parse direction and wrong for the stringify direction, which is the one that actually occurs — so a provider over an untyped backend failed four MUST scenarios while behaving reasonably. They have been moved behind an optional @string-typing capability, with the appendix stating plainly that withholding it is not non-conformance. That is the second capability in that suite whose rule is borrowed rather than specified; @numeric-coercion, from #430, is the first.

flagd wrote a numeric coercion ADR scoped to its own implementations, because nothing normative existed to point at.

Two optional capabilities standing in for one undefined term means conformance reports are not comparable on the point that matters most to an application author: if I ask for a string, do I get a string, or do I get whatever the flag happens to be?

What an answer might look like

Not a proposal, just the shape of the decision:

  1. The canonical definition is authoritative. A provider must know the flag's declared type and raise TYPE_MISMATCH when the accessor does not match it. Clean for applications; requires every provider to have a notion of type its backend may not supply, and makes some existing providers non-conformant.
  2. The provider's own view is authoritative. A provider returns what it has and raises TYPE_MISMATCH only when it cannot produce the requested type at all. Honest about untyped backends; means getStringValue may legitimately never fail, so applications cannot rely on type as a signal.
  3. Say it is provider-defined, normatively. Make the current silence explicit and require providers to document which model they follow. Cheapest, and at least stops each project inventing its own rule silently.

Whichever is chosen, two things would help regardless of the outcome:

  • state whether any requirement obliges a provider to raise TYPE_MISMATCH, since today none does; and
  • if the answer is that coercion is permitted, say whether it must be lossless (which is what #430 asks for numbers, and what the same answer would settle for strings).

Provenance

Raised by @erka while implementing a Flipt provider against the conformance suite in go-sdk-contrib — the suite's string-accessor scenarios failed for a provider that was not doing anything wrong.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with specification/types.md and Requirements 1.3.4 and 2.2.1–2.2.10 in the linked flag-evaluation and providers sections. Compare the competing interpretations with issues #430, #417, and #423, then document the chosen authority for value type, TYPE_MISMATCH, and coercion; the specification should state the resulting normative behavior clearly.

Written by the indexing model from the issue text.

Assessment

Domain
backend-api-design, documentation
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.