aws / aws/aws-toolkit-azure-devops

Task "Output Variables" are not available to Dependent Stages

Open
#446 1 comment 2 reactions 0 assignees View on GitHub
feature-request
Dominant language
TypeScript
Stars
258
Forks
114
PR merge metrics
No merged PRs in 30d

Description

### Summary
When reading the documentation regarding output variables (such as [the Elastic Beanstalk Create App Version task](https://docs.aws.amazon.com/vsts/latest/userguide/elastic-beanstalk-createversion.html#parameters)), I got the impression the variable would be available as other output variables, such as in [this example](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/variables?view=azure-devops&tabs=yaml%2Cbatch#use-outputs-in-a-different-stage) from the Azure documentation. That documentation shows this example:

```YAML
stages:
- stage: One
jobs:
- job: A
steps:
- task: MyTask@1 # this step generates the output variable
name: ProduceVar # because we're going to depend on it, we need to name the step
- stage: Two
- job: B
variables:
# map the output variable from A into this job
varFromA: $[ stageDependencies.One.A.outputs['ProduceVar.MyVar'] ]
steps:
- script: echo $(varFromA) # this step uses the mapped-in variable
```

In this example, the output variable from stage `One`, task `ProduceVar` is available to the second stage through the `stageDependencies` object. I assumed the AWS Toolkit output variables would behave this way, too, given their names as "output variables".

When I ran some tests, however, I found the values passed to the task's output variable was not available to other stages. Here's a simple pipeline I ran to exercise this task:

```YAML
stages:
- stage: build
jobs:
- job: CI_Build
steps:
# a few tasks to build the zipped artifact
- task: BeanstalkCreateApplicationVersion@1
name: createNewEbAppVer
inputs:
awsCredentials: 'AWS CI/CD Admin'
regionName: 'us-east-1'
applicationName: 'my-api'
applicationType: 's3'
deploymentBundleBucket: 'my-bucket'
deploymentBundleKey: 'build/$(Build.BuildId).zip'
outputVariable: 'versionLabel'

- bash: echo "##vso[task.setvariable variable=testStageOutputVar;isOutput=true]this is a stage output var"
name: setVarTest

- task: Bash@3
name: OutputTest
inputs:
targetType: 'inline'
script: |
echo versionLabel: $(versionLabel) # outputs something like 'v1637798886271'
echo buildStageVar: $(buildStageVar) # outputs 'this is a stage output var'

- stage: deploy
dependsOn: build
variables:
buildStageVar: $[ stageDependencies.build.CI_Build.outputs['setVarTest.testStageOutputVar'] ]
versionLabel: $[ stageDependencies.build.CI_Build.outputs['createNewEbAppVer.versionLabel'] ]
jobs:
- job: deploy
steps:
- task: Bash@3
name: OutputTest
inputs:
targetType: 'inline'
script: |
echo versionLabel: $(versionLabel) # outputs ''
echo buildStageVar: $(buildStageVar) # outputs 'this is a stage output var'
```

Notice in the `OutputTest` task, the `versionlabel` variable outputs an empty string. I wanted to understand why. Upon a closer inspection of the `BeanstalkCreateApplicationVersion/TaskOperations` code (see [this line](https://github.com/aws/aws-toolkit-azure-devops/blob/d5306ae1477e5cd6c25d63492e26616a267d2b28/src/tasks/BeanstalkCreateApplicationVersion/TaskOperations.ts#L96)), I found a call to the `azure-pipelines-task-lib/task.js: setVariable` function. In all usages of this function in the AWS Toolkit project, the `isOutput` parameter is not passed, and so it is set to `false` by default. Thus, the task "output variable" is only exposed to the job and is not available to dependent stages.

### Suggestions
I see two possible paths forward:

1. Clarify in [the AWS Toolkit for Azure DevOps user guide](https://docs.aws.amazon.com/vsts/index.html) the true scope of the task output variables. Ensure that future developers know that the output not available to other stages.
2. Modify the various tasks' call to `setVariable` to pass `isOutput = true` so the variable will be available as an output to dependent

### Workaround

Here's a possible workaround to expose the variable to other stages:
```YAML
stages:
- stage: build
jobs:
- job: CI_Build
steps:
# a few tasks to build the zipped artifact
- task: BeanstalkCreateApplicationVersion@1
name: createNewEbAppVer
inputs:
awsCredentials: 'AWS CI/CD Admin'
regionName: 'us-east-1'
applicationName: 'my-api'
applicationType: 's3'
deploymentBundleBucket: 'my-bucket'
deploymentBundleKey: 'build/$(Build.BuildId).zip'
outputVariable: 'versionLabel'

- bash: echo "##vso[task.setvariable variable=versionLabel;isOutput=true]$(versionLabel)"
displayName: 'Set stage output variables'
name: setOutputs

- stage: deploy
dependsOn: build
variables:
versionLabel: $[ stageDependencies.build.CI_Build.outputs['setOutputs.versionLabel'] ]
jobs:
- job: deploy
steps:
- bash: echo 'versionLabel: $(versionLabel)' # outputs something like 'v1637798886271'
name: VariableTest
```

In the above example, the `setOutputs` step loads the local task variable into a stage output variable so it can be referenced in the next stage.

Contributor guide

Open the contributing guide

Research direction

Start with src/tasks/BeanstalkCreateApplicationVersion/TaskOperations.ts around the referenced setVariable call, then inspect other AWS Toolkit usages of setVariable. Compare the task output-variable behavior with the Azure Pipelines stageDependencies example and determine whether the intended fix is documentation or updating task variables; validate the chosen behavior with a dependent-stage pipeline.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, azure, typescript
Domain
ci-cd, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.