Azure / Azure/azure-rest-api-specs

[BUG] Pagination appears broken for threat-intelligence-indicators/list (Sentinel)

Open
#29,403 5 comments 0 reactions 1 assignee Claimed by @xuhumsft View on GitHub
bug customer-reported Mgmt question
Dominant language
TypeSpec
Stars
3.1k
Forks
5.9k
Avg merge
3d 2h
Merged PRs (30d)
424

Description

### API Spec link

https://learn.microsoft.com/en-us/rest/api/securityinsights/threat-intelligence-indicators/list?view=rest-securityinsights-2024-03-01&tabs=HTTP

### API Spec version

2024-03-01

### Describe the bug

The `$top` field is declared as optional, but nowhere does the document state the default value is 100.

When I query the REST endpoint with no `$top` only the first 100 indicators are returned, the subsequent `nextLink`
contains no indicators.

```json
{"value": []}
```

This should be documented in the REST API specs or the behavior should be updated so pagination is not broken when
`$top` is not specified.

### Expected behavior

When `$top` is not specified pagination should continue until all indicators are returned.

### Actual behavior

Only the first 100 indicators are returned, the subsequent `nextLink` contains no indicators.

### Reproduction Steps

// reproduce.py
```python
import os

from typing import Iterator

from azure.identity import ClientSecretCredential
from requests import Session

def fetch(s: Session, *, top: int | None) -> Iterator[dict]:
subscription_id = os.getenv("SENTINEL_SUBSCRIPTION_ID")
resource_group_name = os.getenv("SENTINEL_RESOURCE_GROUP_NAME")
workspace_name = os.getenv("SENTINEL_WORKSPACE_NAME")

r = s.get(
f"https://management.azure.com/subscriptions/{subscription_id}/resourceGroups/{resource_group_name}"
f"/providers/Microsoft.OperationalInsights/workspaces/{workspace_name}/providers/Microsoft.SecurityInsights"
f"/threatIntelligence/main/indicators",
params={"api-version": "2024-03-01", "$top": top},
)
r.raise_for_status()
j = r.json()

yield from j["value"]

while next_link := j.get("nextLink", None):
r = s.get(next_link)
r.raise_for_status()
j = r.json()
yield from j["value"]

def ilen(iterable: Iterator[dict]) -> int:
return sum(1 for _ in iterable)

def main():
secret = ClientSecretCredential(
tenant_id=os.getenv("SENTINEL_TENANT_ID"),
client_id=os.getenv("SENTINEL_CLIENT_ID"),
client_secret=os.getenv("SENTINEL_CLIENT_SECRET"),
)
token = secret.get_token("https://management.azure.com/.default").token

s = Session()
s.headers = {
"Accept": "application/json",
"Authorization": f"bearer {token}",
}

print(f"- {ilen(fetch(s, top=None))=}")
print(f"- {ilen(fetch(s, top=100))=}")
print(f"- {ilen(fetch(s, top=200))=}")
print(f"- {ilen(fetch(s, top=1000))=}")

if __name__ == '__main__':
main()
```

// shell commands
```shell
foo@bar:~$ pip install azure-identity requests
...
foo@bar:~$ python reproduce.py
- ilen(fetch(s, top=None))=100
- ilen(fetch(s, top=100))=100
- ilen(fetch(s, top=200))=200
- ilen(fetch(s, top=1000))=1000
```

### Environment

- Operating System: Linux arch 6.9.3-arch1-1
- Python Version: Python 3.10.14 (venv)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.