Azure / Azure/azure-rest-api-specs

[FEATURE REQ] Allow $expand=instanceView without $filter on VirtualMachines List/ListAll

Open
#42,411 0 comments 0 reactions 0 assignees View on GitHub
Compute customer-reported feature-request Mgmt question Service Attention
Dominant language
TypeSpec
Stars
3.1k
Forks
5.9k
Avg merge
2d 22h
Merged PRs (30d)
444

Description

### API Spec link

https://github.com/Azure/azure-rest-api-specs/blob/main/specification/compute/resource-manager/Microsoft.Compute/Compute/stable/2024-07-01/virtualMachine.json

### API Spec version

2024-07-01

### Please describe the feature.

Howdy folks, hoping this can get some attention, thank you. 🙏

### Summary

Allow `$expand=instanceView` on `VirtualMachines_ListAll` and `VirtualMachines_List` without requiring `$filter`, so that clients can retrieve full VM properties and runtime status in a single API call.

### Problem

Per the published contract, there is no documented way to get full VM properties (`tags`, `location`, `storageProfile`) and runtime status (`instanceView` with power state) in a single VirtualMachines list call for standalone VMs.

The `$expand` parameter on both `VirtualMachines_ListAll` and `VirtualMachines_List` is documented as:

> "The expand expression to apply on operation. 'instanceView' enables fetching run time status of all Virtual Machines, **this can only be specified if a valid $filter option is specified**"

The only documented `$filter` value is `virtualMachineScaleSet/id`, which makes the documented `$expand=instanceView` path VMSS-specific rather than usable for standalone VMs.

The alternative parameter `statusOnly=true` on `VirtualMachines_ListAll` does return `properties.instanceView`, but in our testing strips `tags`, `properties.storageProfile`, and other properties from the response body (see Reproduction below). This omission behavior does not appear to be reflected in the published response schema, the response type is `VirtualMachineListResult` in both cases, with no indication that fields are omitted when `statusOnly` is set.

This forces clients that need subscription-scope list semantics with both VM metadata and power state to either make two ListAll calls and join results client-side, or fall back to per-VM `Get` requests with `$expand=instanceView` (O(N) calls).

### Proposed change: remove `$filter` requirement from `$expand=instanceView`

Allow `$expand=instanceView` on `VirtualMachines_ListAll` and `VirtualMachines_List` without requiring `$filter`. The response would include the existing `VirtualMachine` resource shape with `properties.instanceView` populated.

In the TypeSpec source (`VirtualMachine.tsp`), the `$expand` parameter description would change from:

> "this can only be specified if a valid $filter option is specified"

to indicating that `$filter` is optional when using `$expand`.

In the OpenAPI spec (`virtualMachine.json`), the same description update on both `VirtualMachines_ListAll` and `VirtualMachines_List`.

#### Design considerations

1) Low backward-compatibility risk for existing successful clients
Today, `$expand=instanceView` without `$filter` returns an error (502 on ListAll, 400 on List). In our testing, this combination fails rather than returning data. Enabling it would return data instead of an error. Clients already using `$expand` with `$filter` for VMSS scenarios would not need to change behavior.

2) Subscription-scope fan-out is already demonstrated
The service already supports subscription-scope runtime-status fan-out via `statusOnly=true`, suggesting the scalability concern for retrieving InstanceView across all VMs in a subscription is at least partially addressed. The additional cost would be including the full VM properties alongside the InstanceView data.

3) Likely reduces ARM/RP read pressure
Consolidating two management calls into one would reduce per-subscription read request volume for clients that currently make both a ListAll and a ListAll(statusOnly) call.

4) Payload increase is workload-dependent.
InstanceView includes nested status structures, so the per-VM increase depends on the number of status entries. This would be additive to the existing paginated response.

5) Partial failure handling
In our testing, the `Get` path may return `instanceView: null` when status is unavailable. Azure's own VM power-state documentation notes that with `statusOnly=true`, power state may sometimes be unavailable due to intermittent retrieval issues and recommends retrying. If list operations with `$expand=instanceView` followed the same convention, per-VM status fetch failures could be represented as `instanceView: null` entries without failing the entire page.

Acknowledging potential service-side cost: If there are performance concerns with enabling full-model + InstanceView at subscription scope, a capped or explicitly paginated behavior would still be preferable to the current undocumented split between `statusOnly` and full-resource listing.

Orthogonal to future `$filter` expansion: Removing the `$filter` requirement from `$expand` does not preclude adding new `$filter` values in the future. These are independent features that compose naturally.

### Alternative considered: make `statusOnly=true` return full properties

Making `statusOnly=true` include `tags`, `storageProfile`, and other properties alongside `instanceView` would also address the problem. However, this carries higher backward-compatibility risk: existing clients using `statusOnly=true` may depend on the current response shape or payload size. This would likely need to be gated behind a new API version or a separate parameter.

At minimum, regardless of which approach is taken, the observed behavior of `statusOnly=true` omitting properties should be documented in the spec or response schema, since it is inconsistent with the declared `VirtualMachineListResult` response type.

### Reproduction

Tested against API version `2024-07-01`.

`$expand=instanceView` without `$filter` - ListAll (502):

```http
GET /subscriptions/{sub}/providers/Microsoft.Compute/virtualMachines?api-version=2024-07-01&$expand=instanceView
```

```json
{
"error": {
"code": "ProviderError",
"message": "Resource provider 'Microsoft.Compute' failed to return collection response for type 'virtualMachines'."
}
}
```
`$expand=instanceView` without `$filter` - List per RG (400):

```http
GET /subscriptions/{sub}/resourceGroups/{rg}/providers/Microsoft.Compute/virtualMachines?api-version=2024-07-01&$expand=instanceView
```

```json
{
"error": {
"code": "BadRequest",
"message": "Expand Instance View is only supported when Virtual Machine Scale Set resource filter is applied"
}
}
```

`statusOnly=true` - observed property omission:

```http
GET /subscriptions/{sub}/providers/Microsoft.Compute/virtualMachines?api-version=2024-07-01&statusOnly=true
```

Returns 200 OK. In our testing across 44 VMs in a subscription, every VM in the response had `tags: null` and `properties.storageProfile: null` regardless of actual values, while `properties.instanceView.statuses` was correctly populated. VMs verified to have tags via separate `Get` calls returned `tags: null` through the `statusOnly` path.

### Related issues

- [Azure/azure-sdk-for-go#21682](https://github.com/Azure/azure-sdk-for-go/issues/21682) -> "Get instanceView when listing all VMs" (closed without resolution)
- [Azure/azure-sdk-for-go#6206](https://github.com/Azure/azure-sdk-for-go/issues/6206) -> historical `statusOnly`/`ListAll` behavior issue
- [Azure/azure-sdk-for-go#4828](https://github.com/Azure/azure-sdk-for-go/issues/4828) -> `ListAll` / `InstanceView` history; later comment reports `StorageProfile=nil` when `statusOnly=true`
- [Azure/azure-sdk-for-go#20576](https://github.com/Azure/azure-sdk-for-go/issues/20576) -> adjacent `$expand` confusion on another Compute list endpoint

Contributor guide

Open the contributing guide

Research direction

Start with VirtualMachine.tsp and the 2024-07-01 virtualMachine.json definitions for VirtualMachines_ListAll and VirtualMachines_List. Compare how the TypeSpec source maps to the published OpenAPI descriptions, then check repository guidance for generated specs and validation. Done means the requested contract behavior is clearly represented for both operations and the specification checks pass.

Written by the indexing model from the issue text.

Assessment

Tech stack
openapi
Domain
api, cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.