Custom shell in Windows is having text added to it, in opposition to the description in the docs.
Open
Nobody has claimed this yet.
bug
needs-investigation
Runner Bug
- Dominant language
- C#
- Stars
- 6.3k
- Forks
- 1.4k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 24
Description
Describe the bug
When using a custom shell command, the runner added lines to my script even though the docs say that it wasn't supposed to: https://github.com/actions/runner/blame/feafd3e1d79a627b18e63705d1021165d978ba46/docs/adrs/0277-run-action-shell-options.md#L84
I think that it's due to this function:
I have a custom shell but the first word in my shell is powershell. The function linked above is parsing out that first word. That function is used here: https://github.com/actions/runner/blob/c18c8746db0b7662a13da5596412c05c1ffb07dd/src/Runner.Worker/Handlers/ScriptHandler.cs#L109
Then a function to "fixup" scripts is called: https://github.com/actions/runner/blob/c18c8746db0b7662a13da5596412c05c1ffb07dd/src/Runner.Worker/Handlers/ScriptHandler.cs#L240
And that function only looks at the first word of the shell to determine if the script needs a fixup. The fixup is here: https://github.com/actions/runner/blob/c18c8746db0b7662a13da5596412c05c1ffb07dd/src/Runner.Worker/Handlers/ScriptHandlerHelpers.cs#L56
And that fixup will add the line to the script at the top and the bottom, even for a custom script, so long as it starts with powershell or pwsh.
This is different from what the documentation says. Either the code is wrong or the docs are wrong.
To Reproduce
Steps to reproduce the behavior:
- Use a custom shell as described above. Have that custom shell print out the contents of the script file
- Run it
- See that the file has those extra lines added.
Expected behavior
I expected no extra lines to be added
OS of the machine running the runner? Windows
Job Log Output
Notice that I have a custom shell and that custom shell prints out the contents of the script. Notice that the script has lines added to it.
https://github.com/eyal0/universal-ci-shell/runs/1612490299?check_suite_focus=true
2020-12-27T00:21:29.1418357Z ##[group]Run echo "PKG_CONFIG_PATH=$HOME/.msys_ci_test_path/lib/pkgconfig:$PKG_CONFIG_PATH" >> $GITHUB_ENV
2020-12-27T00:21:29.1419448Z [36;1mecho "PKG_CONFIG_PATH=$HOME/.msys_ci_test_path/lib/pkgconfig:$PKG_CONFIG_PATH" >> $GITHUB_ENV[0m
2020-12-27T00:21:29.1420188Z [36;1mecho "LD_LIBRARY_PATH=$HOME/.msys_ci_test_path/lib:$LD_LIBRARY_PATH" >> $GITHUB_ENV[0m
2020-12-27T00:21:29.1420800Z [36;1mecho "PATH=$HOME/.msys_ci_test_path/bin:$PATH" >> $GITHUB_ENV[0m
2020-12-27T00:21:29.1421337Z [36;1mecho "/msys_ci_test_path/foo" >> $GITHUB_PATH[0m
2020-12-27T00:21:29.1469296Z shell: C:\windows\System32\WindowsPowerShell\v1.0\powershell.EXE -command "[System.Environment]::SetEnvironmentVariable('GITHUB_SCRIPT', ('{0}' -replace '\\','\\')); dir Env:; echo WHAT IS THE SCRIPT CONTENTS; type {0}"
msys2 -c 'echo ORIGINAL SCRIPT BELOW; cat $(cygpath $GITHUB_SCRIPT); echo END OF SCRIPT; if [[ -v MSYSTEM ]]; sed -i 1d $(cygpath $GITHUB_SCRIPT); sed -i \$d $(cygpath $GITHUB_SCRIPT); source $(cygpath $GITHUB_SCRIPT); fi'
2020-12-27T00:21:29.1471002Z env:
2020-12-27T00:21:29.1471339Z MSYSTEM: MINGW64
2020-12-27T00:21:29.1471669Z ##[endgroup]
2020-12-27T00:21:29.3589784Z
2020-12-27T00:21:29.3593419Z Name Value
2020-12-27T00:21:29.3594058Z ---- -----
2020-12-27T00:21:29.3594824Z ALLUSERSPROFILE C:\ProgramData
2020-12-27T00:21:29.3598986Z ANDROID_HOME C:\Program Files (x86)\Android\android-sdk
2020-12-27T00:21:29.3602498Z ANDROID_NDK_HOME C:\Program Files (x86)\Android\android-sdk\ndk-bundle
2020-12-27T00:21:29.3605636Z ANDROID_NDK_PATH C:\Program Files (x86)\Android\android-sdk\ndk-bundle
2020-12-27T00:21:29.3608759Z ANDROID_SDK_ROOT C:\Program Files (x86)\Android\android-sdk
2020-12-27T00:21:29.3613439Z ANT_HOME C:\ProgramData\chocolatey\lib\ant\tools\apache-ant-1.10.9
2020-12-27T00:21:29.3616762Z APPDATA C:\Users\runneradmin\AppData\Roaming
2020-12-27T00:21:29.3620182Z AZURE_EXTENSION_DIR C:\Program Files\Common Files\AzureCliExtensionDirectory
2020-12-27T00:21:29.3623230Z BOOST_ROOT_1_72_0 C:\hostedtoolcache\windows\Boost\1.72.0\x86_64
2020-12-27T00:21:29.3626490Z ChocolateyInstall C:\ProgramData\chocolatey
2020-12-27T00:21:29.3629756Z ChromeWebDriver C:\SeleniumWebDrivers\ChromeDriver
2020-12-27T00:21:29.3632720Z CI true
2020-12-27T00:21:29.3635967Z COBERTURA_HOME C:\cobertura-2.1.1
2020-12-27T00:21:29.3639183Z CommonProgramFiles C:\Program Files\Common Files
2020-12-27T00:21:29.3642348Z CommonProgramFiles(x86) C:\Program Files (x86)\Common Files
2020-12-27T00:21:29.3645549Z CommonProgramW6432 C:\Program Files\Common Files
2020-12-27T00:21:29.3648581Z COMPUTERNAME fv-az153-614
2020-12-27T00:21:29.3653504Z ComSpec C:\windows\system32\cmd.exe
2020-12-27T00:21:29.3654768Z CONDA C:\Miniconda
2020-12-27T00:21:29.3658212Z DEPLOYMENT_BASEPATH C:\actions
2020-12-27T00:21:29.3661443Z DOTNET_MULTILEVEL_LOOKUP 0
2020-12-27T00:21:29.3664685Z DriverData C:\Windows\System32\Drivers\DriverData
2020-12-27T00:21:29.3667807Z EdgeWebDriver C:\SeleniumWebDrivers\EdgeDriver
2020-12-27T00:21:29.3671198Z GCM_INTERACTIVE Never
2020-12-27T00:21:29.3674828Z GeckoWebDriver C:\SeleniumWebDrivers\GeckoDriver
2020-12-27T00:21:29.3677800Z GITHUB_ACTION run2
2020-12-27T00:21:29.3681110Z GITHUB_ACTION_REF
2020-12-27T00:21:29.3684279Z GITHUB_ACTION_REPOSITORY
2020-12-27T00:21:29.3687399Z GITHUB_ACTIONS true
2020-12-27T00:21:29.3690480Z GITHUB_ACTOR eyal0
2020-12-27T00:21:29.3693784Z GITHUB_API_URL https://api.github.com
2020-12-27T00:21:29.3696723Z GITHUB_BASE_REF
2020-12-27T00:21:29.3700171Z GITHUB_ENV D:\a\_temp\_runner_file_commands\set_env_f5e426eb-d357-4ae6-9352-c176c2cb8393
2020-12-27T00:21:29.3703060Z GITHUB_EVENT_NAME push
2020-12-27T00:21:29.3706265Z GITHUB_EVENT_PATH D:\a\_temp\_github_workflow\event.json
2020-12-27T00:21:29.3709534Z GITHUB_GRAPHQL_URL https://api.github.com/graphql
2020-12-27T00:21:29.3712581Z GITHUB_HEAD_REF
2020-12-27T00:21:29.3715711Z GITHUB_JOB test
2020-12-27T00:21:29.3719002Z GITHUB_PATH D:\a\_temp\_runner_file_commands\add_path_f5e426eb-d357-4ae6-9352-c176c2cb8393
2020-12-27T00:21:29.3722035Z GITHUB_REF refs/heads/main
2020-12-27T00:21:29.3725282Z GITHUB_REPOSITORY eyal0/universal-ci-shell
2020-12-27T00:21:29.3728293Z GITHUB_REPOSITORY_OWNER eyal0
2020-12-27T00:21:29.3731450Z GITHUB_RETENTION_DAYS 90
2020-12-27T00:21:29.3734498Z GITHUB_RUN_ID 446236567
2020-12-27T00:21:29.3738204Z GITHUB_RUN_NUMBER 62
2020-12-27T00:21:29.3740949Z GITHUB_SCRIPT D:\\a\\_temp\\6cc3d908-f2d5-41e0-9e9c-863c0831e99c.ps1
2020-12-27T00:21:29.3743932Z GITHUB_SERVER_URL https://github.com
2020-12-27T00:21:29.3747041Z GITHUB_SHA f5638e096c9f722ef6e1c7968cf4cd1ebd48cac9
2020-12-27T00:21:29.3750062Z GITHUB_WORKFLOW CI
2020-12-27T00:21:29.3753401Z GITHUB_WORKSPACE D:\a\universal-ci-shell\universal-ci-shell
2020-12-27T00:21:29.3756307Z GOROOT C:\hostedtoolcache\windows\go\1.14.13\x64
2020-12-27T00:21:29.3759401Z GOROOT_1_10_X64 C:\hostedtoolcache\windows\go\1.10.8\x64
2020-12-27T00:21:29.3762571Z GOROOT_1_11_X64 C:\hostedtoolcache\windows\go\1.11.13\x64
2020-12-27T00:21:29.3765716Z GOROOT_1_12_X64 C:\hostedtoolcache\windows\go\1.12.17\x64
2020-12-27T00:21:29.3768793Z GOROOT_1_13_X64 C:\hostedtoolcache\windows\go\1.13.15\x64
2020-12-27T00:21:29.3771989Z GOROOT_1_14_X64 C:\hostedtoolcache\windows\go\1.14.13\x64
2020-12-27T00:21:29.3775100Z GOROOT_1_15_X64 C:\hostedtoolcache\windows\go\1.15.6\x64
2020-12-27T00:21:29.3778226Z GOROOT_1_9_X64 C:\hostedtoolcache\windows\go\1.9.7\x64
2020-12-27T00:21:29.3781480Z GRADLE_HOME C:\ProgramData\chocolatey\lib\gradle\tools\gradle-6.7
2020-12-27T00:21:29.3784447Z HOMEDRIVE C:
2020-12-27T00:21:29.3787560Z HOMEPATH \Users\runneradmin
2020-12-27T00:21:29.3790784Z IEWebDriver C:\SeleniumWebDrivers\IEDriver
2020-12-27T00:21:29.3794192Z ImageOS win19
2020-12-27T00:21:29.3797405Z ImageVersion 20201210.0
2020-12-27T00:21:29.3800523Z JAVA_HOME C:\Program Files\Java\jdk8u275-b01
2020-12-27T00:21:29.3803643Z JAVA_HOME_11_X64 C:\Program Files\Java\jdk-11.0.9.1+1
2020-12-27T00:21:29.3806689Z JAVA_HOME_13_X64 C:\Program Files\Java\jdk-13.0.2+8
2020-12-27T00:21:29.3809973Z JAVA_HOME_7_X64 C:\Program Files\Java\zulu-7-azure-jdk_7.31.0.5-7.0.232-win_x64
2020-12-27T00:21:29.3813103Z JAVA_HOME_8_X64 C:\Program Files\Java\jdk8u275-b01
2020-12-27T00:21:29.3816235Z LOCALAPPDATA C:\Users\runneradmin\AppData\Local
2020-12-27T00:21:29.3819305Z LOGONSERVER \\fv-az153-614
2020-12-27T00:21:29.3822716Z M2 C:\ProgramData\chocolatey\lib\maven\apache-maven-3.6.3\bin
2020-12-27T00:21:29.3825737Z M2_HOME C:\ProgramData\chocolatey\lib\maven\apache-maven-3.6.3
2020-12-27T00:21:29.3828653Z M2_REPO C:\ProgramData\m2
2020-12-27T00:21:29.3831768Z MAVEN_OPTS -Xms256m
2020-12-27T00:21:29.3835434Z MonAgentClientLocation C:\Packages\Plugins\Microsoft.Azure.Geneva.GenevaMonitoring\2.22.0.1\Monitoring\Agent
2020-12-27T00:21:29.3837933Z MSYSTEM MINGW64
2020-12-27T00:21:29.3841517Z npm_config_cache C:\npm\cache
2020-12-27T00:21:29.3844761Z npm_config_prefix C:\npm\prefix
2020-12-27T00:21:29.3847903Z NUMBER_OF_PROCESSORS 2
2020-12-27T00:21:29.3850966Z OS Windows_NT
2020-12-27T00:21:29.3876019Z Path D:\a\_temp\msys;C:\Users\runneradmin\.dotnet\tools;C:\Program Files\MongoDB\Server\4....
2020-12-27T00:21:29.3879284Z PATHEXT .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC;.CPL
2020-12-27T00:21:29.3882522Z PERFLOG_LOCATION_SETTING RUNNER_PERFLOG
2020-12-27T00:21:29.3885664Z PGBIN C:\Program Files\PostgreSQL\13\bin
2020-12-27T00:21:29.3888788Z PGDATA C:\Program Files\PostgreSQL\13\data
2020-12-27T00:21:29.3891944Z PGPASSWORD root
2020-12-27T00:21:29.3895140Z PGROOT C:\Program Files\PostgreSQL\13
2020-12-27T00:21:29.3898250Z PGUSER postgres
2020-12-27T00:21:29.3901431Z PHPROOT c:\tools\php
2020-12-27T00:21:29.3904592Z PIPX_BIN_DIR C:\Program Files (x86)\pipx_bin
2020-12-27T00:21:29.3907668Z PIPX_HOME C:\Program Files (x86)\pipx
2020-12-27T00:21:29.3917782Z POWERSHELL_DISTRIBUTION_CHA... GitHub-Actions-win19
2020-12-27T00:21:29.3920781Z POWERSHELL_UPDATECHECK Off
2020-12-27T00:21:29.3923845Z PROCESSOR_ARCHITECTURE AMD64
2020-12-27T00:21:29.3927123Z PROCESSOR_IDENTIFIER Intel64 Family 6 Model 85 Stepping 7, GenuineIntel
2020-12-27T00:21:29.3930008Z PROCESSOR_LEVEL 6
2020-12-27T00:21:29.3933143Z PROCESSOR_REVISION 5507
2020-12-27T00:21:29.3936276Z ProgramData C:\ProgramData
2020-12-27T00:21:29.3939634Z ProgramFiles C:\Program Files
2020-12-27T00:21:29.3942565Z ProgramFiles(x86) C:\Program Files (x86)
2020-12-27T00:21:29.3945579Z ProgramW6432 C:\Program Files
2020-12-27T00:21:29.3970070Z PSModulePath C:\Users\runneradmin\Documents\WindowsPowerShell\Modules;C:\Modules\azurerm_2.1.0;C:\...
2020-12-27T00:21:29.3972867Z PUBLIC C:\Users\Public
2020-12-27T00:21:29.3975945Z RTOOLS40_HOME C:\rtools40
2020-12-27T00:21:29.3978997Z RUNNER_OS Windows
2020-12-27T00:21:29.3982162Z RUNNER_PERFLOG C:\actions\perflog
2020-12-27T00:21:29.3985370Z RUNNER_TEMP D:\a\_temp
2020-12-27T00:21:29.3988551Z RUNNER_TOOL_CACHE C:/hostedtoolcache/windows
2020-12-27T00:21:29.3991747Z RUNNER_TRACKING_ID github_836f9fe0-8b41-4a4e-91b6-d48e0c84b37c
2020-12-27T00:21:29.3994788Z RUNNER_WORKSPACE D:\a\universal-ci-shell
2020-12-27T00:21:29.3997774Z SBT_HOME C:\Program Files (x86)\sbt\
2020-12-27T00:21:29.4001778Z SELENIUM_JAR_PATH C:\selenium\selenium-server-standalone.jar
2020-12-27T00:21:29.4005036Z SystemDrive C:
2020-12-27T00:21:29.4008147Z SystemRoot C:\windows
2020-12-27T00:21:29.4011615Z TEMP C:\Users\RUNNER~1\AppData\Local\Temp
2020-12-27T00:21:29.4014805Z TMP C:\Users\RUNNER~1\AppData\Local\Temp
2020-12-27T00:21:29.4017898Z USERDOMAIN fv-az153-614
2020-12-27T00:21:29.4021736Z USERDOMAIN_ROAMINGPROFILE fv-az153-614
2020-12-27T00:21:29.4023744Z USERNAME runneradmin
2020-12-27T00:21:29.4027384Z USERPROFILE C:\Users\runneradmin
2020-12-27T00:21:29.4030641Z VCPKG_INSTALLATION_ROOT C:\vcpkg
2020-12-27T00:21:29.4033940Z VS140COMNTOOLS C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools\
2020-12-27T00:21:29.4036907Z windir C:\windows
2020-12-27T00:21:29.4040142Z WIX C:\Program Files (x86)\WiX Toolset v3.11\
2020-12-27T00:21:29.4894265Z WHAT
2020-12-27T00:21:29.4895110Z IS
2020-12-27T00:21:29.4895492Z THE
2020-12-27T00:21:29.4896129Z SCRIPT
2020-12-27T00:21:29.4896723Z CONTENTS
2020-12-27T00:21:29.4914878Z $ErrorActionPreference = 'stop'
2020-12-27T00:21:29.4915893Z echo "PKG_CONFIG_PATH=$HOME/.msys_ci_test_path/lib/pkgconfig:$PKG_CONFIG_PATH" >> $GITHUB_ENV
2020-12-27T00:21:29.4917129Z echo "LD_LIBRARY_PATH=$HOME/.msys_ci_test_path/lib:$LD_LIBRARY_PATH" >> $GITHUB_ENV
2020-12-27T00:21:29.4917879Z echo "PATH=$HOME/.msys_ci_test_path/bin:$PATH" >> $GITHUB_ENV
2020-12-27T00:21:29.4918762Z echo "/msys_ci_test_path/foo" >> $GITHUB_PATH
2020-12-27T00:21:29.4919340Z
2020-12-27T00:21:29.4920159Z if ((Test-Path -LiteralPath variable:\LASTEXITCODE)) { exit $LASTEXITCODE }
2020-12-27T00:21:29.8027063Z /usr/bin/bash: -c: line 1: syntax error near unexpected token `fi'
2020-12-27T00:21:29.8072513Z
2020-12-27T00:21:29.8073055Z
2020-12-27T00:21:29.8237455Z ##[error]Process completed with exit code 1.
2020-12-27T00:21:29.8305235Z Cleaning up orphan processes
You can see that the shell was custom yet the lines got added anyway.
I found this while working on https://github.com/msys2/setup-msys2/issues/104
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with src/Runner.Worker/Handlers/ScriptHandlerHelpers.cs and ScriptHandler.cs, especially the shell parsing and script fixup functions linked in the report. Read ADR 0277 and run the custom-shell reproduction on Windows, inspecting the generated script. Done means a custom shell beginning with powershell or pwsh does not receive unintended extra lines, or the documentation is corrected to match the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, github-actions, powershell
- Domain
- ci-cd, devtools
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100