microsoft / microsoft/kiota

C#/Dart/PHP integration & idempotency tests fail on live Twitter description (binary unions + non-deterministic generation)

Open
#7,997 2 comments 0 reactions 0 assignees View on GitHub
status:waiting-for-triage type:bug
Dominant language
C#
Stars
3.8k
Forks
333
Avg merge
16h 29m
Merged PRs (30d)
116

Description

### What are you generating using Kiota, clients or plugins?

API Client/SDK

### In what context or format are you using Kiota?

Source Build

### Client library/SDK language

Csharp, Dart, PHP, Java, Ruby, Go, TypeScript

### Describe the bug

The `integration-tests` and `idempotency-tests` workflows (which generate against the live `https://api.twitter.com/2/openapi.json` description) started failing after the upstream Twitter description changed (~2026-07-27). These failures are **not** caused by PR #7995 (which is TypeScript-only and fixes the TS binary-union `TS2358` regression); the last green runs on `main` were 2026-07-23, and the description is fetched live at test time.

There are two distinct failure families:

**1. Generated-code quality for primitive binary unions (integration)** — the Twitter description now contains `media` properties typed as a primitive binary union (e.g. `binary | base64`, both mapping to a byte/binary type). This is the same root construct that #7995 fixes for TypeScript, but the Dart and PHP writers emit analytics-unfriendly code for it:

- **Dart** — `dart analyze` fails on `unnecessary_null_comparison` warnings (treated as errors), e.g.:
```
warning - models/append_media_upload_request_media.dart:15:60 - The operand can't be 'null', so the condition is always 'true'. Remove the condition. - unnecessary_null_comparison
warning - models/chat_media_upload_append_request_media.dart:18:65 - ...
warning - models/media_upload_request_media.dart:15:60 - ...
6 issues found.
```
- **PHP** — PHPStan fails with:
```
Strict comparison using !== between null and null will always evaluate to false. (x3)
... in Microsoft\Kiota\Abstractions\Serialization\ParseNode::getBinaryContent
[ERROR] Found 3 errors
```

**2. Non-deterministic generation (idempotency)** — generating twice from the *same* local description file produces differing output. The reordered content is byte-identical; only the ordering of the `Analytics`/models set differs between runs. **This instability is language-agnostic** and surfaces on a *different* language each CI run (a given run only needs one file to differ to fail). Observed reproductions:

| Language | Differing file |
|---|---|
| C# | `Models/Analytics.cs` |
| Dart | `models/news.dart` |
| Java | `apisdk/models/Analytics.java` |
| Ruby | `models/analytics.rb`, `models/models.rb` |
| Go | `models/analytics.go` |
| TypeScript | `models/index.ts` (barrel/inlined model exports) |

Because it whack-a-moles across languages, the Twitter idempotency leg is now suppressed for **all** languages (`IdempotencySuppressions` `Language: all`) rather than per-language.

### Expected behavior

- Integration: generated Dart/PHP for primitive binary unions passes `dart analyze` / PHPStan with no errors.
- Idempotency: two generations from the same description produce byte-identical output for **all** languages.

### Full breadth of work to complete

This issue is resolved only when **all** of the following are done:

**A. Primitive binary union code quality (integration)** — mirror the TypeScript #7995 fix in the other writers so no null-comparison noise is emitted for `binary | base64` unions:
- [ ] Dart writer — eliminate `unnecessary_null_comparison` for primitive binary union models.
- [ ] PHP writer — eliminate PHPStan strict `!==` between null and null in binary union getters.
- [ ] Audit remaining languages (Go, Java, Python, Ruby, C#, CLI) for the same primitive-binary-union construct and fix any analogous code-quality issues.

**B. Deterministic generation (idempotency)** — make generation order stable so two runs from the same description are byte-identical. This is a **core/shared** fix (the ordering of the models/`Analytics` set), not a per-language patch:
- [ ] Identify and remove the source of non-determinism in model ordering/emission (shared builder/writer traversal — likely an unordered collection or hash-based ordering).
- [ ] Verify byte-identical output across two generations for the Twitter description for **every** language: C#, Dart, Java, Ruby, Go, TypeScript, PHP, Python, CLI.
- [ ] Add/confirm a deterministic-ordering regression test so this cannot silently regress.
- [ ] TypeScript specifically: the model barrel (`models/index.ts`) export/inline ordering must be deterministic.

**C. Cleanup** — once A and B are green, remove the temporary suppressions in `it/config.json`:
- [ ] Remove integration `Suppressions` for `dart` and `php` on the Twitter description.
- [ ] Remove the Twitter `IdempotencySuppressions` `Language: all` entry.
- [ ] Confirm CI is green for integration + idempotency across all languages without suppressions.

### How to reproduce

1. Fetch the current `https://api.twitter.com/2/openapi.json`.
2. Run the integration harness for Dart and PHP: `./it/exec-cmd.ps1` flow (see `integration-tests.yml`).
3. Run the idempotency harness for any language: `./it/compare-generation.ps1 -descriptionUrl -language ` (repeat runs; the failing language varies).

CI evidence:
- integration (dart): https://github.com/microsoft/kiota/actions/runs/30269157821/job/89987842099
- integration (php): https://github.com/microsoft/kiota/actions/runs/30269157821/job/89987838304
- idempotency (csharp): https://github.com/microsoft/kiota/actions/runs/30269161518/job/90072848648
- idempotency (dart): https://github.com/microsoft/kiota/actions/runs/30269161518/job/90072851371
- idempotency (typescript): https://github.com/microsoft/kiota/actions/runs/30859619868/job/91842000911
- idempotency (java): https://github.com/microsoft/kiota/actions/runs/30859619868/job/91841990000
- idempotency (ruby): https://github.com/microsoft/kiota/actions/runs/30859619868/job/91841995658
- idempotency (go): https://github.com/microsoft/kiota/actions/runs/30861589560/job/91845062924

### Open API description file

https://api.twitter.com/2/openapi.json

### Kiota Version

1.34.1 (source build / `main`)

### Latest Kiota version known to work for scenario above?(Not required)

Green on `main` as of 2026-07-23 (#7983); regressed after the live Twitter description changed.

### Known Workarounds

These matrix legs have been suppressed in `it/config.json` so unrelated PRs are not blocked:
- integration `Suppressions` for `dart` and `php`;
- `IdempotencySuppressions` `Language: all` for the Twitter description (collapsed from the earlier per-language entries because the non-determinism surfaces on a different language each run).

Suppressions must be removed once this issue is resolved (see Cleanup above).

### Other information

Split of concerns:
- Dart/PHP (and any other affected languages) integration = writer changes needed to emit clean code for primitive binary unions (mirror of the TypeScript #7995 fix).
- Idempotency = language-agnostic non-deterministic ordering of the `Analytics`/models set; needs a shared/core deterministic-ordering fix and must be validated for every language.

Contributor guide

Open the contributing guide

Research direction

Start with it/exec-cmd.ps1, it/compare-generation.ps1, integration-tests.yml, and it/config.json, then trace the shared builder/writer traversal and the Dart and PHP writers. Reproduce the Twitter failures and compare repeated generations across languages. Done means clean integration checks, byte-identical output, regression coverage, and removal of the listed suppressions.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, dart, go, java, openapi, php, ruby, typescript
Domain
build-system, ci-cd, devtools, testing-qa, tooling
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.