Objects are unexpectedly accumulated when a CimCmdlet or CDXML-based command is in the middle of a pipeline

Prerequisites

• Write a descriptive title.
• Make sure you are able to repro it on the latest released version
• Search the existing issues.
• Refer to the FAQ.
• Refer to Differences between Windows PowerShell 5.1 and PowerShell.

Summary

When a CimCmdlet cmdlet or CDXML function is in the middle of a pipeline (not first/last), the following appears to occur:

• Objects processed by the CimCmdlet/CDXML command are unexpectedly accumulated in the middle of the pipeline.
• Once all objects have been processed in the middle of the pipeline, EndProcessing is called on the CimCmdlet/CDXML command and any upstream commands.
• Downstream commands finally start to receive input.

Related:

• https://github.com/PowerShell/PowerShell/issues/20546

Steps to reproduce

$cn = [pscustomobject] @{ ClassName = 'Win32_ComputerSystem' }
$c = [Console]

1, 2, 3 |
    ForEach-Object -Begin { $c::WriteLine('B1') } -Process { $c::WriteLine('P1'); $cn } -End { $c::WriteLine('E1') } |
    Get-CimInstance |
    ForEach-Object -Begin { $c::WriteLine('B2') } -Process { $c::WriteLine('P2') } -End { $c::WriteLine('E2') }

# B1
# B2
# P1
# P1
# P1 <-- Objects accumulated instead of passing through
# E1 <-- EndProcessing prematurely called
# P2 <-- Finally, objects pass through to downstream commands
# P2
# P2
# E2

ParameterBinding Information: 0 : BIND arg [Win32_ComputerSystem] to param [ClassName] SUCCESSFUL
[...]
ParameterBinding Information: 0 : BIND arg [Win32_ComputerSystem] to param [ClassName] SUCCESSFUL
[...]
ParameterBinding Information: 0 : BIND arg [Win32_ComputerSystem] to param [ClassName] SUCCESSFUL
[...]
ParameterBinding Information: 0 : CALLING EndProcessing <--- Write-Output
ParameterBinding Information: 0 : CALLING EndProcessing <--- ForEach-Object (#1)
ParameterBinding Information: 0 : CALLING EndProcessing <--- Get-CimInstance
ParameterBinding Information: 0 : BIND PIPELINE object to parameters: [ForEach-Object]
[...]
ParameterBinding Information: 0 : CALLING EndProcessing <--- ForEach-Object (#2)

Impact

This fundamentally breaks PowerShell's item-by-item processing when the middle of a pipeline involves a CimCmdlet/CDXML command. For example, short-circuiting with Select-Object -First cannot be performed. Notice how commands upstream from Get-Disk must process all objects before Select-Object can raise its stop processing exception.

Get-Partition | 
    ForEach-Object { [Console]::WriteLine('1'); $_ } | 
    Get-Disk | 
    ForEach-Object { [Console]::WriteLine('2'); $_ } | 
    Select-Object -First 1

# 1
# 1
# 1
# 1
# 1
# 1
# 2

-PipelineVariable is broken in the process. The exact behavior depends on the type and position of the CimCmdlet/CDXML command.

• Downstream contains CimCmdlet/CDXML command:

  # Applicable when the first command is not a CDXML command. 
  # The first command *can* be a CimCmdlet command.
  Write-Output 1 2 3 -pv var |
      ForEach-Object { [Console]::WriteLine("1: $var"); [pscustomobject] @{ Number = 0 } } |
      Get-Disk |
      ForEach-Object { [Console]::WriteLine("2: $var") }
  
  # 1: 1
  # 1: 2
  # 1: 3
  # 2: 3 <-- PV remains as the last accumulated object
  # 2: 3
  # 2: 3
  

• Downstream contains CimCmdlet/CDXML AND the first command is CDXML:

  Get-Partition -pv var |
      ForEach-Object { [Console]::WriteLine("1: $($var.PartitionNumber)"); $_ } |
      Get-Disk |
      ForEach-Object { [Console]::WriteLine("2: $($var.PartitionNumber)") }
  
  # 1: 1
  # 1: 2
  # 1: 3
  # 1: 4
  # 1: 5
  # 1: 6
  # 2:   <-- PV is reset to $null
  # 2:
  # 2:
  # 2:
  # 2:
  # 2:
  

  The above scenario is the subject of https://github.com/PowerShell/PowerShell/issues/20546.

Expected behavior

• Objects flow through the pipeline one-by-one from start to end instead of being unexpectedly accumulated.
• EndProcessing is not called prematurely on upstream commands in the middle of a pipeline.

B1
B2
P1
P2
P1
P2
P1
P2
E1
E2

Actual behavior

B1
B2
P1
P1
P1
E1
P2
P2
P2
E2

Environment data

Name                           Value
----                           -----
PSVersion                      7.6.0-preview.3
PSEdition                      Core
GitCommitId                    7.6.0-preview.3
OS                             Microsoft Windows 10.0.19045
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0, 5.0, 5.1…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Start by running the provided CimCmdlet/CDXML pipeline reproducer and compare its output with the expected item-by-item sequence. Use the included Trace-Command excerpt to follow EndProcessing and pipeline binding; done means objects reach downstream commands one at a time without premature upstream EndProcessing.