C#/Dart/PHP integration & idempotency tests fail on live Twitter description (binary unions + non-deterministic generation)
- Dominant language
- C#
- Stars
- 3.8k
- Forks
- 333
- Avg merge
- 16h 29m
- Merged PRs (30d)
- 116
Description
### What are you generating using Kiota, clients or plugins?
API Client/SDK
### In what context or format are you using Kiota?
Source Build
### Client library/SDK language
Csharp, Dart, PHP, Java, Ruby, Go, TypeScript
### Describe the bug
The `integration-tests` and `idempotency-tests` workflows (which generate against the live `https://api.twitter.com/2/openapi.json` description) started failing after the upstream Twitter description changed (~2026-07-27). These failures are **not** caused by PR #7995 (which is TypeScript-only and fixes the TS binary-union `TS2358` regression); the last green runs on `main` were 2026-07-23, and the description is fetched live at test time.
There are two distinct failure families:
**1. Generated-code quality for primitive binary unions (integration)** — the Twitter description now contains `media` properties typed as a primitive binary union (e.g. `binary | base64`, both mapping to a byte/binary type). This is the same root construct that #7995 fixes for TypeScript, but the Dart and PHP writers emit analytics-unfriendly code for it:
- **Dart** — `dart analyze` fails on `unnecessary_null_comparison` warnings (treated as errors), e.g.:
```
warning - models/append_media_upload_request_media.dart:15:60 - The operand can't be 'null', so the condition is always 'true'. Remove the condition. - unnecessary_null_comparison
warning - models/chat_media_upload_append_request_media.dart:18:65 - ...
warning - models/media_upload_request_media.dart:15:60 - ...
6 issues found.
```
- **PHP** — PHPStan fails with:
```
Strict comparison using !== between null and null will always evaluate to false. (x3)
... in Microsoft\Kiota\Abstractions\Serialization\ParseNode::getBinaryContent
[ERROR] Found 3 errors
```
**2. Non-deterministic generation (idempotency)** — generating twice from the *same* local description file produces differing output. The reordered content is byte-identical; only the ordering of the `Analytics`/models set differs between runs. **This instability is language-agnostic** and surfaces on a *different* language each CI run (a given run only needs one file to differ to fail). Observed reproductions:
| Language | Differing file |
|---|---|
| C# | `Models/Analytics.cs` |
| Dart | `models/news.dart` |
| Java | `apisdk/models/Analytics.java` |
| Ruby | `models/analytics.rb`, `models/models.rb` |
| Go | `models/analytics.go` |
| TypeScript | `models/index.ts` (barrel/inlined model exports) |
Because it whack-a-moles across languages, the Twitter idempotency leg is now suppressed for **all** languages (`IdempotencySuppressions` `Language: all`) rather than per-language.
### Expected behavior
- Integration: generated Dart/PHP for primitive binary unions passes `dart analyze` / PHPStan with no errors.
- Idempotency: two generations from the same description produce byte-identical output for **all** languages.
### Full breadth of work to complete
This issue is resolved only when **all** of the following are done:
**A. Primitive binary union code quality (integration)** — mirror the TypeScript #7995 fix in the other writers so no null-comparison noise is emitted for `binary | base64` unions:
- [ ] Dart writer — eliminate `unnecessary_null_comparison` for primitive binary union models.
- [ ] PHP writer — eliminate PHPStan strict `!==` between null and null in binary union getters.
- [ ] Audit remaining languages (Go, Java, Python, Ruby, C#, CLI) for the same primitive-binary-union construct and fix any analogous code-quality issues.
**B. Deterministic generation (idempotency)** — make generation order stable so two runs from the same description are byte-identical. This is a **core/shared** fix (the ordering of the models/`Analytics` set), not a per-language patch:
- [ ] Identify and remove the source of non-determinism in model ordering/emission (shared builder/writer traversal — likely an unordered collection or hash-based ordering).
- [ ] Verify byte-identical output across two generations for the Twitter description for **every** language: C#, Dart, Java, Ruby, Go, TypeScript, PHP, Python, CLI.
- [ ] Add/confirm a deterministic-ordering regression test so this cannot silently regress.
- [ ] TypeScript specifically: the model barrel (`models/index.ts`) export/inline ordering must be deterministic.
**C. Cleanup** — once A and B are green, remove the temporary suppressions in `it/config.json`:
- [ ] Remove integration `Suppressions` for `dart` and `php` on the Twitter description.
- [ ] Remove the Twitter `IdempotencySuppressions` `Language: all` entry.
- [ ] Confirm CI is green for integration + idempotency across all languages without suppressions.
### How to reproduce
1. Fetch the current `https://api.twitter.com/2/openapi.json`.
2. Run the integration harness for Dart and PHP: `./it/exec-cmd.ps1` flow (see `integration-tests.yml`).
3. Run the idempotency harness for any language: `./it/compare-generation.ps1 -descriptionUrl -language ` (repeat runs; the failing language varies).
CI evidence:
- integration (dart): https://github.com/microsoft/kiota/actions/runs/30269157821/job/89987842099
- integration (php): https://github.com/microsoft/kiota/actions/runs/30269157821/job/89987838304
- idempotency (csharp): https://github.com/microsoft/kiota/actions/runs/30269161518/job/90072848648
- idempotency (dart): https://github.com/microsoft/kiota/actions/runs/30269161518/job/90072851371
- idempotency (typescript): https://github.com/microsoft/kiota/actions/runs/30859619868/job/91842000911
- idempotency (java): https://github.com/microsoft/kiota/actions/runs/30859619868/job/91841990000
- idempotency (ruby): https://github.com/microsoft/kiota/actions/runs/30859619868/job/91841995658
- idempotency (go): https://github.com/microsoft/kiota/actions/runs/30861589560/job/91845062924
### Open API description file
https://api.twitter.com/2/openapi.json
### Kiota Version
1.34.1 (source build / `main`)
### Latest Kiota version known to work for scenario above?(Not required)
Green on `main` as of 2026-07-23 (#7983); regressed after the live Twitter description changed.
### Known Workarounds
These matrix legs have been suppressed in `it/config.json` so unrelated PRs are not blocked:
- integration `Suppressions` for `dart` and `php`;
- `IdempotencySuppressions` `Language: all` for the Twitter description (collapsed from the earlier per-language entries because the non-determinism surfaces on a different language each run).
Suppressions must be removed once this issue is resolved (see Cleanup above).
### Other information
Split of concerns:
- Dart/PHP (and any other affected languages) integration = writer changes needed to emit clean code for primitive binary unions (mirror of the TypeScript #7995 fix).
- Idempotency = language-agnostic non-deterministic ordering of the `Analytics`/models set; needs a shared/core deterministic-ordering fix and must be validated for every language.
Contributor guide
Research direction
Start with it/exec-cmd.ps1, it/compare-generation.ps1, integration-tests.yml, and it/config.json, then trace the shared builder/writer traversal and the Dart and PHP writers. Reproduce the Twitter failures and compare repeated generations across languages. Done means clean integration checks, byte-identical output, regression coverage, and removal of the listed suppressions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, dart, go, java, openapi, php, ruby, typescript
- Domain
- build-system, ci-cd, devtools, testing-qa, tooling
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100