GoogleCloudPlatform / GoogleCloudPlatform/cloud-builders-community

Windows builder slow file copy

Open
#347 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
1.3k
Forks
848
PR merge metrics
No merged PRs in 30d

Description

## Affected builder image
`gcr.io/cloud-builders-community/windows`

## Expected Behavior
When the builder copies files from the workspace on the Cloud Build agent to the Windows VM (based on Windows Remote Management protocol), it should quickly copy a large workspace of 2000 directories and 100 MB of data. It should take a few minutes.

## Actual Behavior
The build times out after 2 hours of trying to copy data.

## Steps to Reproduce the Problem
N/A

## Additional Info
The package used to copy files is slow: https://github.com/GoogleCloudPlatform/cloud-builders-community/blob/master/windows-builder/builder/builder/remote.go#L12
It sequentially processes files, breaks each file into 8KB chunks, and performs 3 HTTP round trips for each such chunk.
This implementation of WinRM-based file copying is hundreds of times faster: https://github.com/jbrekelmans/winrm (disclaimer I am the author).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.