aws / aws/aws-toolkit-azure-devops
Task "Output Variables" are not available to Dependent Stages
- Dominant language
- TypeScript
- Stars
- 258
- Forks
- 114
- PR merge metrics
- No merged PRs in 30d
Description
### Summary
When reading the documentation regarding output variables (such as [the Elastic Beanstalk Create App Version task](https://docs.aws.amazon.com/vsts/latest/userguide/elastic-beanstalk-createversion.html#parameters)), I got the impression the variable would be available as other output variables, such as in [this example](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#use-outputs-in-a-different-stage) from the Azure documentation. That documentation shows this example:
```YAML
stages:
- stage: One
jobs:
- job: A
steps:
- task: MyTask@1 # this step generates the output variable
name: ProduceVar # because we're going to depend on it, we need to name the step
- stage: Two
- job: B
variables:
# map the output variable from A into this job
varFromA: $[ stageDependencies.One.A.outputs['ProduceVar.MyVar'] ]
steps:
- script: echo $(varFromA) # this step uses the mapped-in variable
```
In this example, the output variable from stage `One`, task `ProduceVar` is available to the second stage through the `stageDependencies` object. I assumed the AWS Toolkit output variables would behave this way, too, given their names as "output variables".
When I ran some tests, however, I found the values passed to the task's output variable was not available to other stages. Here's a simple pipeline I ran to exercise this task:
```YAML
stages:
- stage: build
jobs:
- job: CI_Build
steps:
# a few tasks to build the zipped artifact
- task: BeanstalkCreateApplicationVersion@1
name: createNewEbAppVer
inputs:
awsCredentials: 'AWS CI/CD Admin'
regionName: 'us-east-1'
applicationName: 'my-api'
applicationType: 's3'
deploymentBundleBucket: 'my-bucket'
deploymentBundleKey: 'build/$(Build.BuildId).zip'
outputVariable: 'versionLabel'
- bash: echo "##vso[task.setvariable variable=testStageOutputVar;isOutput=true]this is a stage output var"
name: setVarTest
- task: Bash@3
name: OutputTest
inputs:
targetType: 'inline'
script: |
echo versionLabel: $(versionLabel) # outputs something like 'v1637798886271'
echo buildStageVar: $(buildStageVar) # outputs 'this is a stage output var'
- stage: deploy
dependsOn: build
variables:
buildStageVar: $[ stageDependencies.build.CI_Build.outputs['setVarTest.testStageOutputVar'] ]
versionLabel: $[ stageDependencies.build.CI_Build.outputs['createNewEbAppVer.versionLabel'] ]
jobs:
- job: deploy
steps:
- task: Bash@3
name: OutputTest
inputs:
targetType: 'inline'
script: |
echo versionLabel: $(versionLabel) # outputs ''
echo buildStageVar: $(buildStageVar) # outputs 'this is a stage output var'
```
Notice in the `OutputTest` task, the `versionlabel` variable outputs an empty string. I wanted to understand why. Upon a closer inspection of the `BeanstalkCreateApplicationVersion/TaskOperations` code (see [this line](https://github.com/aws/aws-toolkit-azure-devops/blob/d5306ae1477e5cd6c25d63492e26616a267d2b28/src/tasks/BeanstalkCreateApplicationVersion/TaskOperations.ts#L96)), I found a call to the `azure-pipelines-task-lib/task.js: setVariable` function. In all usages of this function in the AWS Toolkit project, the `isOutput` parameter is not passed, and so it is set to `false` by default. Thus, the task "output variable" is only exposed to the job and is not available to dependent stages.
### Suggestions
I see two possible paths forward:
1. Clarify in [the AWS Toolkit for Azure DevOps user guide](https://docs.aws.amazon.com/vsts/index.html) the true scope of the task output variables. Ensure that future developers know that the output not available to other stages.
2. Modify the various tasks' call to `setVariable` to pass `isOutput = true` so the variable will be available as an output to dependent
### Workaround
Here's a possible workaround to expose the variable to other stages:
```YAML
stages:
- stage: build
jobs:
- job: CI_Build
steps:
# a few tasks to build the zipped artifact
- task: BeanstalkCreateApplicationVersion@1
name: createNewEbAppVer
inputs:
awsCredentials: 'AWS CI/CD Admin'
regionName: 'us-east-1'
applicationName: 'my-api'
applicationType: 's3'
deploymentBundleBucket: 'my-bucket'
deploymentBundleKey: 'build/$(Build.BuildId).zip'
outputVariable: 'versionLabel'
- bash: echo "##vso[task.setvariable variable=versionLabel;isOutput=true]$(versionLabel)"
displayName: 'Set stage output variables'
name: setOutputs
- stage: deploy
dependsOn: build
variables:
versionLabel: $[ stageDependencies.build.CI_Build.outputs['setOutputs.versionLabel'] ]
jobs:
- job: deploy
steps:
- bash: echo 'versionLabel: $(versionLabel)' # outputs something like 'v1637798886271'
name: VariableTest
```
In the above example, the `setOutputs` step loads the local task variable into a stage output variable so it can be referenced in the next stage.
Contributor guide
Research direction
Start with src/tasks/BeanstalkCreateApplicationVersion/TaskOperations.ts around the referenced setVariable call, then inspect other AWS Toolkit usages of setVariable. Compare the task output-variable behavior with the Azure Pipelines stageDependencies example and determine whether the intended fix is documentation or updating task variables; validate the chosen behavior with a dependent-stage pipeline.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, azure, typescript
- Domain
- ci-cd, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100