Azure / Azure/azure-rest-api-specs
[BUG] Pagination appears broken for threat-intelligence-indicators/list (Sentinel)
- Dominant language
- TypeSpec
- Stars
- 3.1k
- Forks
- 5.9k
- Avg merge
- 3d 2h
- Merged PRs (30d)
- 424
Description
### API Spec link
https://learn.microsoft.com/en-us/rest/api/securityinsights/threat-intelligence-indicators/list?view=rest-securityinsights-2024-03-01&tabs=HTTP
### API Spec version
2024-03-01
### Describe the bug
The `$top` field is declared as optional, but nowhere does the document state the default value is 100.
When I query the REST endpoint with no `$top` only the first 100 indicators are returned, the subsequent `nextLink`
contains no indicators.
```json
{"value": []}
```
This should be documented in the REST API specs or the behavior should be updated so pagination is not broken when
`$top` is not specified.
### Expected behavior
When `$top` is not specified pagination should continue until all indicators are returned.
### Actual behavior
Only the first 100 indicators are returned, the subsequent `nextLink` contains no indicators.
### Reproduction Steps
// reproduce.py
```python
import os
from typing import Iterator
from azure.identity import ClientSecretCredential
from requests import Session
def fetch(s: Session, *, top: int | None) -> Iterator[dict]:
subscription_id = os.getenv("SENTINEL_SUBSCRIPTION_ID")
resource_group_name = os.getenv("SENTINEL_RESOURCE_GROUP_NAME")
workspace_name = os.getenv("SENTINEL_WORKSPACE_NAME")
r = s.get(
f"https://management.azure.com/subscriptions/{subscription_id}/resourceGroups/{resource_group_name}"
f"/providers/Microsoft.OperationalInsights/workspaces/{workspace_name}/providers/Microsoft.SecurityInsights"
f"/threatIntelligence/main/indicators",
params={"api-version": "2024-03-01", "$top": top},
)
r.raise_for_status()
j = r.json()
yield from j["value"]
while next_link := j.get("nextLink", None):
r = s.get(next_link)
r.raise_for_status()
j = r.json()
yield from j["value"]
def ilen(iterable: Iterator[dict]) -> int:
return sum(1 for _ in iterable)
def main():
secret = ClientSecretCredential(
tenant_id=os.getenv("SENTINEL_TENANT_ID"),
client_id=os.getenv("SENTINEL_CLIENT_ID"),
client_secret=os.getenv("SENTINEL_CLIENT_SECRET"),
)
token = secret.get_token("https://management.azure.com/.default").token
s = Session()
s.headers = {
"Accept": "application/json",
"Authorization": f"bearer {token}",
}
print(f"- {ilen(fetch(s, top=None))=}")
print(f"- {ilen(fetch(s, top=100))=}")
print(f"- {ilen(fetch(s, top=200))=}")
print(f"- {ilen(fetch(s, top=1000))=}")
if __name__ == '__main__':
main()
```
// shell commands
```shell
foo@bar:~$ pip install azure-identity requests
...
foo@bar:~$ python reproduce.py
- ilen(fetch(s, top=None))=100
- ilen(fetch(s, top=100))=100
- ilen(fetch(s, top=200))=200
- ilen(fetch(s, top=1000))=1000
```
### Environment
- Operating System: Linux arch 6.9.3-arch1-1
- Python Version: Python 3.10.14 (venv)
Contributor guide
Assessment
This issue has not been assessed yet.