Touch should open files with FILE_WRITE_ATTRIBUTES on .NET Framework
- Dominant language
- C#
- Stars
- 5.5k
- Forks
- 1.5k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 133
Description
### Issue Description
On MSBuild running atop .NET Framework, the `Touch` task opens the target file with `GENERIC_WRITE | FILE_READ_ATTRIBUTES`.
Windows will deny opening a file this way when there are other handles to the file open without `FILE_SHARE_WRITE`. Thus, `Touch` can fail with a sharing violation against readers that are would otherwise be compatible with the file's attributes being updated.
Instead, Touch should open the file with `FILE_WRITE_ATTRIBUTES` like it does on .NET (Core).
MSBuild can collide with itself: `ProjectImportsCollector` embeds `EmbedInBinlog` files into the binlog asynchronously on a background thread, opening them with `FILE_SHARE_READ | FILE_SHARE_DELETE`. That work can still be in flight after the target that produced the file has finished, so later `Touch` of the same file can intermittently fail.
### Steps to Reproduce
1. Run MSBuild.exe (.NET Framework) with `-bl`, in a build where a target adds generated files to the `EmbedInBinlog` item.
2. Have a subsequent task in the same target `Touch` one of those just-embedded files.
My case involves NuGet `Restore` (via the `Microsoft.Build.NoTargets` SDK) as a preamble to building `.vcxproj` files. The build-in `Restore` target adds the generated `project.assets.json`, `.props`, and `.targets` to `EmbedInBinlog`. Immediately afterwards my `RestoreCpp` target `Touch`es the generated `.g.props` file, because NuGet does not always rewrite the generated files even when inputs change, which causes my downstream up-to-date checks to fail:
```xml
```
The failure is timing-dependent, so it does not reproduce on every build.
A binlog and an ETW/WPA trace (File I/O provider) capturing the failure are available on request.
### Expected Behavior
`Touch` succeeds. Timestamps are updated.
### Actual Behavior
```
error MSB3374: The last access/last write time on file
"...\restore\RestoreCppPackageReferences.proj.nuget.g.props" cannot be set.
The process cannot access the file '...\restore\RestoreCppPackageReferences.proj.nuget.g.props'
because it is being used by another process.
```
In my WPA File I/O trace, I see `Touch`'s `GENERIC_WRITE` open being denied against a concurrent read-only handle, and that handle closing 0.2 ms later.
Conflicting file operations with stack traces
TimeProcessOperationFlagsResultFile objectStack
30.606914200MSBuild.exe (7576)Create
FILE_OPEN : FILE_SYNCHRONOUS_IO_NONALERT FILE_NON_DIRECTORY_FILE FILE_OPEN_NO_RECALL : FILE_SHARE_READ FILE_SHARE_DELETE
Success0xFFFF8480062B76A0
ntdll.dll!RtlUserThreadStart
kernel32.dll!BaseThreadInitThunk
clr.dll!Thread::intermediateThreadProc
clr.dll!ThreadpoolMgr::WorkerThreadStart
clr.dll!ThreadpoolMgr::ExecuteWorkRequest
clr.dll!ManagedPerAppDomainTPCount::DispatchWorkItem
clr.dll!ManagedThreadBase_FullTransitionWithAD
clr.dll!ManagedThreadBase_DispatchOuter
clr.dll!ManagedThreadBase_DispatchMiddle
clr.dll!ManagedThreadBase_DispatchInner
clr.dll!QueueUserWorkItemManagedCallback
clr.dll!MethodDescCallSite::CallTargetWorker
clr.dll!CallDescrWorkerWithHandler
clr.dll!CallDescrWorkerInternal
mscorlib.ni.dll!System.Threading.ThreadPoolWorkQueue.Dispatch()
mscorlib.ni.dll!System.Threading.Tasks.Task.ExecuteEntry(Boolean)
mscorlib.ni.dll!System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef)
mscorlib.ni.dll!System.Threading.ExecutionContext.Run(...)
mscorlib.ni.dll!System.Threading.ExecutionContext.RunInternal(...)
mscorlib.ni.dll!System.Threading.Tasks.Task.Execute()
Microsoft.Build.dll!Microsoft.Build.Logging.ProjectImportsCollector+<>c__DisplayClass16_0::<AddFileHelper>g__TryAddFile|0
Microsoft.Build.dll!Microsoft.Build.Logging.ProjectImportsCollector::AddFileCore
mscorlib.ni.dll!System.IO.FileStream..ctor(System.String, System.IO.FileMode, System.IO.FileAccess, System.IO.FileShare)
mscorlib.ni.dll!System.IO.FileStream.Init(...)
mscorlib.ni.dll!Microsoft.Win32.Win32Native.SafeCreateFile(...)
mscorlib.ni.dll!DomainNeutralILStubClass.IL_STUB_PInvoke(...)
KernelBase.dll!CreateFileW
KernelBase.dll!CreateFileInternal
ntdll.dll!ZwCreateFile
ntoskrnl.exe!KiSystemServiceCopyEnd
ntoskrnl.exe!NtCreateFile
ntoskrnl.exe!IopCreateFile
ntoskrnl.exe!ObOpenObjectByNameEx
ntoskrnl.exe!ObpLookupObjectName
ntoskrnl.exe!IopParseDevice
ntoskrnl.exe!IofCallDriver
FLTMGR.SYS!FltpCreate
FLTMGR.SYS!FltpPassThroughInternal
FLTMGR.SYS!FltpPerformPreCallbacksWorker
fileinfo.sys!FIPreCreateCallback
ntoskrnl.exe!EtwpTraceFileIo
ntoskrnl.exe!EtwpLogKernelEvent
ntoskrnl.exe!EtwpTraceStackWalk
30.607410000MSBuild.exe (19400)Create
FILE_OPEN : FILE_SYNCHRONOUS_IO_NONALERT FILE_NON_DIRECTORY_FILE FILE_OPEN_NO_RECALL : FILE_SHARE_READ FILE_SHARE_WRITE
A file cannot be opened because the share access flags are incompatible. (0xc0000043)0xFFFF84800874A160
ntdll.dll!RtlUserThreadStart
kernel32.dll!BaseThreadInitThunk
clr.dll!Thread::intermediateThreadProc
clr.dll!ThreadNative::KickOffThread
clr.dll!ManagedThreadBase_FullTransitionWithAD
clr.dll!ManagedThreadBase_DispatchOuter
clr.dll!ManagedThreadBase_DispatchMiddle
clr.dll!ManagedThreadBase_DispatchInner
clr.dll!ThreadNative::KickOffThread_Worker
clr.dll!MethodDescCallSite::CallTargetWorker
clr.dll!CallDescrWorkerWithHandler
clr.dll!CallDescrWorkerInternal
mscorlib.ni.dll!System.Threading.ThreadHelper.ThreadStart()
mscorlib.ni.dll!System.Threading.ExecutionContext.Run(...)
mscorlib.ni.dll!System.Threading.ExecutionContext.RunInternal(...)
Microsoft.Build.dll!Microsoft.Build.BackEnd.RequestBuilder+DedicatedThreadsTaskScheduler::<InjectThread>b__6_0
mscorlib.ni.dll!System.Threading.Tasks.Task.ExecuteEntry(Boolean)
mscorlib.ni.dll!System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef)
mscorlib.ni.dll!System.Threading.Tasks.Task.Execute()
mscorlib.ni.dll!System.Threading.Tasks.TaskSchedulerAwaitTaskContinuation+<>c.<Run>b__2_0(System.Object)
mscorlib.ni.dll!System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run()
mscorlib.ni.dll!System.Threading.ExecutionContext.Run(...)
mscorlib.ni.dll!System.Threading.ExecutionContext.RunInternal(...)
Microsoft.Build.dll!Microsoft.Build.BackEnd.TargetEntry+<ProcessBucket>d__51::MoveNext
Microsoft.Build.dll!Microsoft.Build.BackEnd.TaskBuilder::ExecuteTask
mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[System.__Canon]::Start
Microsoft.Build.dll!Microsoft.Build.BackEnd.TaskBuilder+<ExecuteTask>d__13::MoveNext
Microsoft.Build.dll!Microsoft.Build.BackEnd.TaskBuilder::ExecuteTask
mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[System.__Canon]::Start
Microsoft.Build.dll!Microsoft.Build.BackEnd.TaskBuilder+<ExecuteTask>d__18::MoveNext
Microsoft.Build.dll!Microsoft.Build.BackEnd.TaskBuilder::ExecuteBucket
mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[System.__Canon]::Start
Microsoft.Build.dll!Microsoft.Build.BackEnd.TaskBuilder+<ExecuteBucket>d__19::MoveNext
Microsoft.Build.dll!Microsoft.Build.BackEnd.TaskBuilder::InitializeAndExecuteTask
mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[System.__Canon]::Start
Microsoft.Build.dll!Microsoft.Build.BackEnd.TaskBuilder+<InitializeAndExecuteTask>d__24::MoveNext
Microsoft.Build.dll!Microsoft.Build.BackEnd.TaskBuilder::ExecuteInstantiatedTask
mscorlib.dll!System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[System.__Canon]::Start
Microsoft.Build.dll!Microsoft.Build.BackEnd.TaskBuilder+<ExecuteInstantiatedTask>d__26::MoveNext
Microsoft.Build.dll!Microsoft.Build.BackEnd.TaskExecutionHost::Execute
Microsoft.Build.Tasks.Core.dll!Microsoft.Build.Tasks.Touch::Execute
Microsoft.Build.Tasks.Core.dll!Microsoft.Build.Tasks.Touch::ExecuteImpl
Microsoft.Build.Tasks.Core.dll!Microsoft.Build.Tasks.Touch::TouchFile
mscorlib.ni.dll!System.IO.File.SetLastAccessTime(System.String, System.DateTime)
mscorlib.ni.dll!System.IO.File.SetLastAccessTimeUtc(System.String, System.DateTime)
mscorlib.ni.dll!System.IO.File.OpenFile(System.String, System.IO.FileAccess, Microsoft.Win32.SafeHandles.SafeFileHandle ByRef)
mscorlib.ni.dll!System.IO.FileStream..ctor(System.String, System.IO.FileMode, System.IO.FileAccess, System.IO.FileShare, Int32)
mscorlib.ni.dll!System.IO.FileStream.Init(...)
mscorlib.ni.dll!Microsoft.Win32.Win32Native.SafeCreateFile(...)
mscorlib.ni.dll!DomainNeutralILStubClass.IL_STUB_PInvoke(...)
KernelBase.dll!CreateFileW
KernelBase.dll!CreateFileInternal
ntdll.dll!ZwCreateFile
ntoskrnl.exe!KiSystemServiceCopyEnd
ntoskrnl.exe!NtCreateFile
ntoskrnl.exe!IopCreateFile
ntoskrnl.exe!ObOpenObjectByNameEx
ntoskrnl.exe!ObpLookupObjectName
ntoskrnl.exe!IopParseDevice
ntoskrnl.exe!IofCallDriver
FLTMGR.SYS!FltpCreate
FLTMGR.SYS!FltpPassThroughInternal
FLTMGR.SYS!FltpPerformPreCallbacksWorker
fileinfo.sys!FIPreCreateCallback
ntoskrnl.exe!EtwpTraceFileIo
ntoskrnl.exe!EtwpLogKernelEvent
ntoskrnl.exe!EtwpTraceStackWalk
30.607640100MSBuild.exe (7576)Close
Success0xFFFF8480062B76A0
ntdll.dll!RtlUserThreadStart
kernel32.dll!BaseThreadInitThunk
clr.dll!Thread::intermediateThreadProc
clr.dll!ThreadpoolMgr::WorkerThreadStart
clr.dll!ThreadpoolMgr::ExecuteWorkRequest
clr.dll!ManagedPerAppDomainTPCount::DispatchWorkItem
clr.dll!ManagedThreadBase_FullTransitionWithAD
clr.dll!ManagedThreadBase_DispatchOuter
clr.dll!ManagedThreadBase_DispatchMiddle
clr.dll!ManagedThreadBase_DispatchInner
clr.dll!QueueUserWorkItemManagedCallback
clr.dll!MethodDescCallSite::CallTargetWorker
clr.dll!CallDescrWorkerWithHandler
clr.dll!CallDescrWorkerInternal
mscorlib.ni.dll!System.Threading.ThreadPoolWorkQueue.Dispatch()
mscorlib.ni.dll!System.Threading.Tasks.Task.ExecuteEntry(Boolean)
mscorlib.ni.dll!System.Threading.Tasks.Task.ExecuteWithThreadLocal(System.Threading.Tasks.Task ByRef)
mscorlib.ni.dll!System.Threading.ExecutionContext.Run(...)
mscorlib.ni.dll!System.Threading.ExecutionContext.RunInternal(...)
mscorlib.ni.dll!System.Threading.Tasks.Task.Execute()
Microsoft.Build.dll!Microsoft.Build.Logging.ProjectImportsCollector+<>c__DisplayClass16_0::<AddFileHelper>g__TryAddFile|0
Microsoft.Build.dll!Microsoft.Build.Logging.ProjectImportsCollector::AddFileCore
Microsoft.Build.dll!Microsoft.Build.Logging.ProjectImportsCollector::AddFileCore
mscorlib.ni.dll!System.IO.Stream.Close()
mscorlib.ni.dll!System.IO.FileStream.Dispose(Boolean)
mscorlib.ni.dll!System.IO.FileStream.Dispose(Boolean)
clr.dll!SafeHandle::DisposeNative
clr.dll!SafeHandle::Dispose
clr.dll!SafeHandle::Release
clr.dll!SafeHandle::RunReleaseMethod
clr.dll!DispatchCallSimple
clr.dll!CallDescrWorkerWithHandler
clr.dll!CallDescrWorkerInternal
mscorlib.ni.dll!System.Threading.Thread.Finalize()
KernelBase.dll!CloseHandle
ntdll.dll!ZwClose
ntoskrnl.exe!KiSystemServiceCopyEnd
ntoskrnl.exe!NtClose
ntoskrnl.exe!ObCloseHandleTableEntry
ntoskrnl.exe!ObfDereferenceObjectWithTag
ntoskrnl.exe!ObpRemoveObjectRoutine
ntoskrnl.exe!IopDeleteFile
ntoskrnl.exe!IofCallDriver
FLTMGR.SYS!FltpDispatch
FLTMGR.SYS!FltpPassThrough
FLTMGR.SYS!FltpPassThroughInternal
FLTMGR.SYS!FltpPerformPreCallbacksWorker
fileinfo.sys!FIPreCloseCallback
ntoskrnl.exe!EtwpTraceFileIo
ntoskrnl.exe!EtwpLogKernelEvent
ntoskrnl.exe!EtwpTraceStackWalk
### Analysis
MSBuild's `Touch.TouchFile` calls `File.SetLastAccessTime` / `File.SetLastWriteTime` ([`src/Tasks/Touch.cs:171-172`](https://github.com/dotnet/msbuild/blob/main/src/Tasks/Touch.cs#L171-L172)).
On .NET Framework, `File.SetLastAccessTimeUtc` calls `File.OpenFile(path, FileAccess.Write, out handle)`, which maps to `GENERIC_WRITE`.
`Touch` on .NET Framework should stop relying on `File.SetLastAccessTime` / `File.SetLastWriteTime` and instead open the file itself with `dwDesiredAccess = FILE_WRITE_ATTRIBUTES` (plus `FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE`) and set the timestamps via `SetFileInformationByHandle(FileBasicInfo, …)`. That is what .NET (Core) already does, so the fix converges the two platforms' behavior. It requires calling `CreateFile` more manually than the Framework BCL does.
Secondarily, `ProjectImportsCollector` could open embedded files with `FILE_SHARE_WRITE` as well, since it only reads them. There is also no way today for a target to wait for another target's binlog-embedding background work to drain, so this race cannot be ordered around in project code.
## Workaround
For my particular case, I ended up using a dedicated sentinel file to track whether restore had been done instead of re-using the generated .proj.nuget.g.props.
A different workaround would be to use a `RoslynCodeTaskFactory` task that re-implements the code logic of Touch, doing exactly the `FILE_WRITE_ATTRIBUTES` + `SetFileInformationByHandle` sequence described above.
UpdateFileLastWriteTime workaround task
```xml
```
## .NET (Core) implementation of `File.SetLastAccessTimeUtc`
.NET's `File.SetLastAccessTimeUtc` calls `CreateFile` with `FILE_WRITE_ATTRIBUTES` and full sharing, then sets the timestamps with `SetFileInformationByHandle`.
.NET's call chain from [`dotnet/runtime@f7d2d52`](https://github.com/dotnet/runtime/tree/f7d2d521667ad056e7dfbe3fc1c89cb9597d2d7b)
1. [`File.SetLastAccessTimeUtc`](https://github.com/dotnet/runtime/blob/f7d2d521667ad056e7dfbe3fc1c89cb9597d2d7b/src/libraries/System.Private.CoreLib/src/System/IO/File.cs#L353-L354) calls `FileSystem.SetLastAccessTime`:
```csharp
public static void SetLastAccessTimeUtc(string path, DateTime lastAccessTimeUtc)
=> FileSystem.SetLastAccessTime(Path.GetFullPath(path), GetUtcDateTimeOffset(lastAccessTimeUtc), false);
```
2. [`FileSystem.SetLastAccessTime`](https://github.com/dotnet/runtime/blob/f7d2d521667ad056e7dfbe3fc1c89cb9597d2d7b/src/libraries/System.Private.CoreLib/src/System/IO/FileSystem.Windows.cs#L511-L512) calls `SetFileTime`:
```csharp
public static void SetLastAccessTime(string fullPath, DateTimeOffset time, bool asDirectory)
=> SetFileTime(fullPath, asDirectory, lastAccessTime: time.ToFileTime());
```
3. [`SetFileTime`](https://github.com/dotnet/runtime/blob/f7d2d521667ad056e7dfbe3fc1c89cb9597d2d7b/src/libraries/System.Private.CoreLib/src/System/IO/FileSystem.Windows.cs#L474-L483) obtains the handle from `OpenHandleToWriteAttributes`:
```csharp
using SafeFileHandle handle = OpenHandleToWriteAttributes(fullPath, asDirectory);
SetFileTime(handle, fullPath, creationTime, lastAccessTime, lastWriteTime);
```
4. [`OpenHandleToWriteAttributes`](https://github.com/dotnet/runtime/blob/f7d2d521667ad056e7dfbe3fc1c89cb9597d2d7b/src/libraries/System.Private.CoreLib/src/System/IO/FileSystem.Windows.cs#L226-L231) opens with `FILE_WRITE_ATTRIBUTES`:
```csharp
SafeFileHandle handle = Interop.Kernel32.CreateFile(
fullPath,
Interop.Kernel32.FileOperations.FILE_WRITE_ATTRIBUTES,
FileShare.ReadWrite | FileShare.Delete,
FileMode.Open,
dwFlagsAndAttributes);
```
5. The timestamps are then set via [`SetFileInformationByHandle` with `FileBasicInfo`](https://github.com/dotnet/runtime/blob/f7d2d521667ad056e7dfbe3fc1c89cb9597d2d7b/src/libraries/System.Private.CoreLib/src/System/IO/FileSystem.Windows.cs#L492-L502):
```csharp
var basicInfo = new Interop.Kernel32.FILE_BASIC_INFO
{
CreationTime = creationTime,
LastAccessTime = lastAccessTime,
LastWriteTime = lastWriteTime,
};
if (!Interop.Kernel32.SetFileInformationByHandle(fileHandle, Interop.Kernel32.FileBasicInfo, &basicInfo, (uint)sizeof(Interop.Kernel32.FILE_BASIC_INFO)))
{
throw Win32Marshal.GetExceptionForLastWin32Error(fullPath ?? fileHandle.Path);
}
```
### Versions & Configurations
Initially encountered on MSBuild version 17.14.40+3e7442088 running on a x64 Windows 11 machine. MSBuild is invoked directly on a .slnx file.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.