CircleCI-Public / CircleCI-Public/windows-preview-docs

store_test_artifacts fails

Open
#7 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
9
Forks
1
PR merge metrics
No merged PRs in 30d

Description

I have a job running on win preview that is coded in config.yml like this:

```
test-36:
parallelism: 2
working_directory: c:\cache
executor: win/preview-default
environment:
BUILD_PREFIX: 3.6.3
TEST_FOLDER: ./tests

steps:
- checkout
- run: choco install python --version=3.6.3
- run: choco install checksum
- run:
name: Setup Python Version
command: |
$env:Path += ";C:\Python36;C:\Python36\Scripts"
python --version >> _tmp_file_python_version 2>&1
echo $CIRCLE_JOB >> _tmp_file_python_version 2>&1
cat _tmp_file_python_version 2>&1
- restore_cache: *restore_cache
- run:
name: Install Requirements
command: |
$env:Path += ";C:\Python36;C:\Python36\Scripts"
if (-not (Test-Path "venv")) {
pip install virtualenv
virtualenv venv
venv\Scripts\activate

$client = New-Object System.Net.WebClient
$client.DownloadFile("https://bootstrap.pypa.io/get-pip.py", "c:\tmp\get-pip.py")
python c:\tmp\get-pip.py
Remove-Item c:\tmp\get-pip.py -Force

python --version
pip --version

pip install --requirement dev-requirements.txt --upgrade --progress-bar off --ignore-installed
}

# We install the latest on top of the venv in order to get the latest packages
venv\Scripts\activate
pip install --requirement dev-requirements.txt --upgrade --progress-bar off
- save_cache: *save_cache
- run:
name: tests
command: |
$env:Path += ";C:\Python36;C:\Python36\Scripts"
venv\Scripts\activate
(Get-Command python).Path
mkdir -p workspace\test_artifacts\$env:BUILD_PREFIX\coverage
mkdir -p workspace\test_results\$env:BUILD_PREFIX
cmd /c mklink /d test_artifact workspace\test_artifacts\$env:BUILD_PREFIX\coverage
cmd /c mklink /d test_results workspace\test_results\$env:BUILD_PREFIX
nosetests --with-xunit --with-coverage --cover-erase --cover-branches --cover-package=. --cover-xml --cover-html `
--cover-html-dir=workspace/test_artifacts/$env:BUILD_PREFIX/coverage `
--cover-xml-file=workspace/test_results/$env:BUILD_PREFIX/coverage-report.xml `
--xunit-file=workspace/test_results/$env:BUILD_PREFIX/nosetests.xml `
$env:TEST_FOLDER
- store_test_results:
path: test_results

- store_artifacts:
path: test_artifact

- persist_to_workspace:
root: workspace
paths:
- test_artifact\*
- test_results\*
```

it actually succeeds. however, the `store_artifacts` step fails with this error:

```
Uploading c:\cache\test_artifact to cache/test_artifact
Uploading c:\cache\test_artifact (12 kB): FAILED with error MultipartUpload: upload multipart failed
upload id: dsxtW66mIwGlXDFMA0mlG0XJ0aBj1Oymg1iMwahDfyAbOeSAebShvRtY7ohpl28ECr_omwffagHti1Su4aV7.d0FibUKoB8zi6TQygyThqAT_h1HlnJONuQnLvKpNB0xS_buvcttQqOY917.iZRGjg--
caused by: BodyHashError: failed to compute body hashes
caused by: read c:\cache\test_artifact: The handle is invalid.
```

You can see it at https://circleci.com/gh/missinglinkai/ml-core/1782 - uploading artifacts step

It _may_ have something to do with the fact that `test_artifacts` is a symbolic link. However, `test_results` in the next step is also a symbolic link and it works fine.

The actual path exists and has files inside, verified by rerunning with ssh:

```
PS C:\Users\circleci> cd c:\cache
cd c:\cache
PS C:\cache> dir
dir

Directory: C:\cache

Mode LastWriteTime Length Name
---- ------------- ------ ----
d----- 7/11/2019 11:36 AM .circleci
d----- 7/11/2019 11:36 AM .github
d----- 7/11/2019 11:36 AM git_hooks
d----- 7/11/2019 11:37 AM missinglink
d----- 7/11/2019 11:36 AM scripts
d----- 7/11/2019 11:37 AM tests
d----l 7/11/2019 11:37 AM test_artifact
d----l 7/11/2019 11:37 AM test_results
d----- 7/10/2019 1:46 PM venv
d----- 7/11/2019 11:37 AM workspace
-a---- 7/11/2019 11:36 AM 755 .codeclimate.yml
-a---- 7/11/2019 11:38 AM 95558 .coverage
-a---- 7/11/2019 11:36 AM 130 .coveragerc
-a---- 7/11/2019 11:36 AM 304 .editorconfig
-a---- 7/11/2019 11:36 AM 1403 .gitignore
-a---- 7/11/2019 11:36 AM 599 .releaserc.json
-a---- 7/11/2019 11:36 AM 188 dev-requirements.txt
-a---- 7/11/2019 11:36 AM 117 requirements.txt
-a---- 7/11/2019 11:36 AM 99 setup.cfg
-a---- 7/11/2019 11:36 AM 1385 setup.py
-a---- 7/11/2019 11:37 AM 30 _tmp_file_python_version

PS C:\cache> dir test_artifact
dir test_artifact

Directory: C:\cache\test_artifact

Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 7/11/2019 11:38 AM 18458 coverage_html.js
-a---- 7/11/2019 11:38 AM 9644 index.html
-a---- 7/11/2019 11:38 AM 731 jquery.ba-throttle-debounce.min.js
-a---- 7/11/2019 11:38 AM 3065 jquery.hotkeys.js
-a---- 7/11/2019 11:38 AM 1502 jquery.isonscreen.js
-a---- 7/11/2019 11:38 AM 95785 jquery.min.js
-a---- 7/11/2019 11:38 AM 12795 jquery.tablesorter.min.js
-a---- 7/11/2019 11:38 AM 112 keybd_closed.png
-a---- 7/11/2019 11:38 AM 112 keybd_open.png
-a---- 7/11/2019 11:38 AM 66595 missinglink_core_api_py.html
-a---- 7/11/2019 11:38 AM 55921 missinglink_core_avro_utils_py.html
-a---- 7/11/2019 11:38 AM 76767 missinglink_core_config_py.html
-a---- 7/11/2019 11:38 AM 15013 missinglink_core_context_py.html
-a---- 7/11/2019 11:38 AM 17896 missinglink_core_default_params_py.html
-a---- 7/11/2019 11:38 AM 4707 missinglink_core_eprint_py.html
-a---- 7/11/2019 11:38 AM 5626 missinglink_core_exceptions_py.html
-a---- 7/11/2019 11:38 AM 30533 missinglink_core_json_utils_py.html
-a---- 7/11/2019 11:38 AM 84113 missinglink_core_multi_process_control_py.html
-a---- 7/11/2019 11:38 AM 18454 missinglink_core_print_status_py.html
-a---- 7/11/2019 11:38 AM 6748 missinglink_core_update_timer_py.html
-a---- 7/11/2019 11:38 AM 7549 missinglink_core_utils_file_utils_py.html
-a---- 7/11/2019 11:38 AM 19189 missinglink_core_utils_os_utils_py.html
-a---- 7/11/2019 11:38 AM 2790 missinglink_core_utils___init___py.html
-a---- 7/11/2019 11:38 AM 3062 missinglink_core___init___py.html
-a---- 7/11/2019 11:38 AM 4181 missinglink___init___py.html
-a---- 7/11/2019 11:38 AM 3646 status.json
-a---- 7/11/2019 11:38 AM 6757 style.css
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the Windows preview job from config.yml, focusing on the store_artifacts step and its test_artifact symbolic link. Compare it with the succeeding store_test_results step and inspect the reported BodyHashError. Done means the artifact directory uploads successfully without the invalid-handle failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
powershell, python
Domain
ci-cd, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.