Azure-Samples / Azure-Samples/azure-batch-samples

CSharp/GettingStarted/02_PoolsAndResourceFiles throws NotFound for stdout.txt when run a 2nd time

Open
#291 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
268
Forks
478
PR merge metrics
No merged PRs in 30d

Description

### Expected behavior
Project can be run multiple times without needing to manually modify batch pools.

### Actual behavior
Project run only works once. All subsequent runs throw an exception.

I've tried with various pool sizes, but even with 1 node it throws the exception.

### Reproduction steps
1. Run the project once. The pool gets created, then resized and everything works as expected.
2. Run the project a 2nd time and an exception is thrown: "NotFound". The exception occurs when attempting to get access to the stdout.txt file:

https://github.com/Azure-Samples/azure-batch-samples/blob/ff3dcdcf0c75909ecb20c87a3a6802e6d13ba239/CSharp/Common/GettingStartedCommon.cs#L126

### Work around
1. Using Batch Explorer delete the pool.
2. Wait until pool is totally deleted.
3. Run the project. It functions as expected.

### Example failed output
```
Running with the following settings:
-------------------------------------
PoolId = PoolsAndResourceFilesPool
PoolTargetNodeCount = 1
PoolOSFamily = 5
PoolNodeVirtualMachineSize = standard_d1_v2
ShouldDeletePool = False
ShouldDeleteJob = True
BlobContainer = PoolAndResourceFilesContainer

BatchAccountName = xxxxxxxxxxxxxxxxxx
BatchAccountKey = xxxxxxxxxxxxxxxxxxx==
BatchServiceUrl = xxxxxxxxxxxxxxxxxxx
StorageAccountName = xxxxxxxxxxxxxxxx
StorageAccountKey = xxxxxxxxxxxxxxxxx==
StorageServiceUrl = core.windows.net

Uploading resources to storage container: poolandresourcefilescontainer
Uploading C:\Users\xxxx\AppData\Local\Temp\StartTask.txt to https://xxxxxxxx.blob.core.windows.net/poolandresourcefilescontainer/StartTask.txt
Done uploading C:\Users\xxxx\AppData\Local\Temp\StartTask.txt
Attempting to create pool: PoolsAndResourceFilesPool
The pool already existed when we tried to create it
Listing Jobs
============
State of job SimpleJob-xxxx-20200510-145405 is Active
============
Listing Pools
=============
State of pool PoolsAndResourceFilesPool is Active and it has 1 dedicated nodes and 0 low-priority nodes of size standard_d1_v2
=============
Task task_with_file1
Deleting container: mab-2020-05-10-12-54-06c1cf5a24-c2ca-4e0e-ad65-041314fdfd22
Deleting job: SimpleJob-xxxx-20200510-145405
Microsoft.Azure.Batch.Common.BatchException: Operation returned an invalid status code 'NotFound' ---> Microsoft.Azure.Batch.Protocol.Models.BatchErrorException: Operation returned an invalid status code 'NotFound'
at Microsoft.Azure.Batch.Protocol.FileOperations.d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.Batch.Protocol.BatchRequestBase`2.d__42.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.Azure.Batch.Protocol.BatchRequestBase`2.d__40.MoveNext()
--- End of inner exception stack trace ---
at Microsoft.Azure.Batch.Samples.Common.GettingStartedCommon.d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at Microsoft.Azure.Batch.Samples.PoolsAndResourceFiles.JobSubmitter.d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.Azure.Batch.Samples.PoolsAndResourceFiles.JobSubmitter.d__5.MoveNext()
Request Information
ClientRequestId:
RequestId:1ffe597d-792d-4085-8c6b-16b54535a9c4
HttpStatusCode:NotFound
StatusMessage:The specified file does not exist.

Unhandled Exception: System.AggregateException: One or more errors occurred. ---> Microsoft.Azure.Batch.Common.BatchException: Operation returned an invalid status code 'NotFound' ---> Microsoft.Azure.Batch.Protocol.Models.BatchErrorException: Operation returned an invalid status code 'NotFound'
at Microsoft.Azure.Batch.Protocol.FileOperations.d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.Azure.Batch.Protocol.BatchRequestBase`2.d__42.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.Azure.Batch.Protocol.BatchRequestBase`2.d__40.MoveNext()
--- End of inner exception stack trace ---
at Microsoft.Azure.Batch.Samples.Common.GettingStartedCommon.d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
at Microsoft.Azure.Batch.Samples.PoolsAndResourceFiles.JobSubmitter.d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.Azure.Batch.Samples.PoolsAndResourceFiles.JobSubmitter.d__5.MoveNext()
--- End of inner exception stack trace ---
at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)
at System.Threading.Tasks.Task.Wait()
at Microsoft.Azure.Batch.Samples.PoolsAndResourceFiles.Program.Main(String[] args)

D:\Repos\Azure\AZ203\azure-batch-samples\CSharp\GettingStarted\02_PoolsAndResourceFiles\JobSubmitter\bin\Debug\net462\JobSubmitter.exe (process 49684) exited with code 0.
To automatically close the console when debugging stops, enable Tools->Options->Debugging->Automatically close the console when debugging stops.
Press any key to close this window . . .

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.