dotnet / dotnet/workload-versions

Hashing calculation doesn't work on large workloads

Open
#437 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Shell
Stars
14
Forks
22
Avg merge
2d 7h
Merged PRs (30d)
38

Description

## Summary

Currently, the `mono` packs workload doesn't properly calculate the unique hash value that we use when creating a unique URL for the VS drop upload. This causes the (biggest) workload to be uploaded multiple times (per servicing branch count). It will emit this many, many times in the build log:

```
Exception calling "Write" with "1" argument(s): "Exception of type 'System.OutOfMemoryException' was thrown."
At D:\a\_work\1\s\eng\create-workload-drops.ps1:35 char:76
+ ... t-Content -Encoding ASCII -Raw | ForEach-Object { $writer.Write($_) }
+ ~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], MethodInvocationException
+ FullyQualifiedErrorId : OutOfMemoryException
```

Basically, the algo is collecting the contents of all of the files in the workload, then creating the hash. This obviously doesn't work if there is too much content in a single workload. The algo would need to be changed so that:
- Each file is individually hashed
- Then calculate the hash of hashes

If this change is done, it needs to be done on all branches. Otherwise, different calculations on different branches would invalidate the point of the hash.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.