Azure-Samples / Azure-Samples/ai-model-start
DeepSeek-R1-0528: reasoning_tokens count always 0; reasoning item emission intermittent
- Dominant language
- Bicep
- Stars
- 3
- Forks
- 4
- PR merge metrics
- No merged PRs in 30d
Description
### Summary
When calling `DeepSeek-R1-0528` via the Responses API on a Foundry project endpoint, the structured `reasoning` output item is now emitted with `summary[].text` populated (great!), but two server-side issues remain:
1. **`reasoning_tokens` is always 0** even when summary text is clearly present
2. **Reasoning item emission is intermittent** — roughly half of responses to the same prompt return only a `message` item, no `reasoning` item
This makes it hard to (a) build accurate token-accounting/billing logic and (b) write reliable demos or tests that depend on the reasoning surface.
### Reproduction
```python
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from openai import OpenAI
endpoint = ""
client = OpenAI(
base_url=endpoint.rstrip("/") + "/openai/v1",
api_key=get_bearer_token_provider(DefaultAzureCredential(), "https://ai.azure.com/.default"),
)
for i in range(4):
r = client.responses.create(
model="DeepSeek-R1-0528",
input="What's heavier: a pound of feathers or a pound of bricks? Think step by step.",
max_output_tokens=2000,
)
types = [it.type for it in r.output]
rt = r.usage.output_tokens_details.reasoning_tokens
print(f"run {i+1}: items={types} reasoning_tokens={rt}")
```
Observed (4 runs, same prompt and parameters):
```
run 1: items=['message'] reasoning_tokens=0
run 2: items=['reasoning', 'message'] reasoning_tokens=0
run 3: items=['message'] reasoning_tokens=0
run 4: items=['message'] reasoning_tokens=0
```
When the `reasoning` item is present, `summary[0].text` contains a multi-paragraph chain-of-thought summary. But `reasoning_tokens` stays at 0 in every case.
### Expected behavior
- `reasoning_tokens` should reflect the number of tokens consumed by the model's reasoning trace (the way it does for OpenAI o-series models) — so billing/quota tracking is accurate
- The `reasoning` output item should be emitted consistently for the same prompt/parameters (or there should be a documented way to force it on, e.g. `include=["reasoning"]`)
### Impact
- Apps using the universal walk-output-items pattern can't reliably show the reasoning summary to users (it appears ~50% of the time)
- Apps tracking compute cost see `reasoning_tokens=0` and may significantly under-report DeepSeek usage
- Demos and tutorials (like https://github.com/Azure-Samples/ai-model-start) need to caveat the behavior in code comments
### Environment
- Foundry account in East US 2
- `DeepSeek-R1-0528` deployment, version 1
- `openai==2.36.0`, `azure-identity==1.x`
- Project endpoint, `/openai/v1` base URL, EntraID `DefaultAzureCredential` with `https://ai.azure.com/.default` scope
### Cross-reference
PR #11 surfaces reasoning summaries across all 5 language examples and handles both the present and absent cases gracefully, but this server-side fix would let the templates be deterministic and the token accounting accurate.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.