PowerShell / PowerShell/PowerShell

Out-File not reporting error on failure to write to UNC network path when stream is being repeatedly opened and closed.

Open
#21,545 2 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Needs-Triage
Dominant language
C#
Stars
55.5k
Forks
8.5k
Avg merge
1d 2h
Merged PRs (30d)
88

Description

Prerequisites
Steps to reproduce

I am trying to extract a representative sample of a 200MB csv file by writing the header and every 500th row to a new file for testers to use. I relied on out-file -append to add each row matching the modulus condition to the destination file on network share, but what I found is that I had slightly fewer rows in the sample file than expected. Repeated runs produced slightly different numbers of rows in the destination file (expected 2014, actual ranged between 1992-2011). I did not get an terminating error.

The behaviour does NOT occur if I use a location on my local hard drive for the destination file.

$destfile = "\\UNCSHARE\Folder\Export_Sample_$(get-date -Format "yyyyMMdd_HHmmss").txt"

$Original = get-content "\\UNCSHARE\Folder\200MB_Export_20231208_1545.txt"

[int64]$ln = 0
[int64]$SampleCount = 0

foreach ($line in $Original) {
    $ln++
    if ($ln -eq 1 -or $ln % 500 -eq 0) {
        $line | Out-File -FilePath $destfile -Append -ErrorAction Stop
        $SampleCount++
    }
}
write-host $SampleCount

(get-content $destfile).count 

I tried a suggested alternative of switching to the native API call:

#$line | Out-File -FilePath $destfile -Append -ErrorAction Stop -Encoding utf8
[System.IO.File]::AppendAllText($destfile, "$line`n")

That still doesn't produce a terminating error in Powershell 7.4.2, but does in Windows Powershell 5.1

Exception calling "AppendAllText" with "2" argument(s): "The process cannot access the file 
'\\UNCSHARE\Export_sample20240424_164810.txt' because it is being used 
by another process."

Out-File to a UNC path doesn't produce a terminating error in Powershell 7.4.2 or 5.1.

Expected behavior
For my example file. 
write-host $SampleCount
--2014
(get-content $destfile).count 
--2014
Actual behavior
For my example file. 
write-host $SampleCount
--2014
(get-content $destfile).count 
--a number ranging between 1995 and 2011 (depending on the number of times it failed to write a single line to the destination)
Error details

No response

Environment data
Name                           Value
----                           -----
PSVersion                      7.4.2
PSEdition                      Core
GitCommitId                    7.4.2
OS                             Microsoft Windows 10.0.19045
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0
Visuals

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the repeated Out-File -Append writes against a UNC path on Windows, comparing them with a local destination and System.IO.File::AppendAllText. Trace how Out-File handles stream opening, closing, and write failures, then verify that failed writes are surfaced as errors and no sample rows are silently lost.

Written by the indexing model from the issue text.

Assessment

Tech stack
powershell
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.