(3.8.0 - 3.9.3) ParallelCluster Build Image Failing during Installation of Minitar Ruby Gem Dependency
- Dominant language
- Python
- Stars
- 888
- Forks
- 314
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 43
Description
## The issue
During `pcluster build-image`, ParallelCluster installs Berkshelf gems for dependency management. `Minitar` is a dependency of Berkshelf. On Tuesday, August 6, Minitar released 2 different versions, v1.0.0 and last version of minitar v0.12 with the deprecated archive-tar-minitar v0.12.
The latest minitar v1.0.0 removed the `Archive::Tar` namespace and `archive/tar` path as they have been deprecated since 2017. And moved the executable in a separate `minitar-cli` package. These changes in `minitar` v 1.0.0+ causes failure in Berkshelf installation and hence the ParallelCluster Build Image fails.
Reference to Berkshelf Installation Failure https://github.com/chef/berkshelf/issues/26
## Error details
ParallelCluster fails to build images. The error surfaced by ParallelCluster Build Image is:
```
pcluster describe-image --image-id $IMAGE_ID | jq ".imagebuilderImageStatusReason"
Error occurred during operation 'Workflow Execution ID: 'wf-xxxxxx-xxxxxx-xxxxxx-xxxxxx-xxxxxx' failed with reason: Document arn::imagebuilder:::component/parallelclusterimage-*//1 failed!'
```
And, below is the snippet of the failure in ParallelCluster Build Image Logs which you can capture using [pcluster export-image-logs - AWS ParallelCluster](https://docs.aws.amazon.com/parallelcluster/latest/ug/pcluster.export-image-logs-v3.html)
```
LANG=en_US.UTF-8 sudo /opt/cinc/embedded/bin/berks vendor /etc/chef/cookbooks --delete || (echo 'Vendoring cookbook failed.' && exit 1)
done;
:85:in `require': cannot load such file -- archive/tar/minitar (LoadError)
from :85:in `require'
from /opt/cinc/embedded/lib/ruby/gems/3.1.0/gems/berkshelf-8.0.7/lib/berkshelf/packager.rb:1:in `'
from /opt/cinc/embedded/lib/ruby/gems/3.1.0/gems/berkshelf-8.0.7/lib/berkshelf/berksfile.rb:1:in `require_relative'
from /opt/cinc/embedded/lib/ruby/gems/3.1.0/gems/berkshelf-8.0.7/lib/berkshelf/berksfile.rb:1:in `'
from /opt/cinc/embedded/lib/ruby/gems/3.1.0/gems/berkshelf-8.0.7/lib/berkshelf.rb:222:in `require_relative'
from /opt/cinc/embedded/lib/ruby/gems/3.1.0/gems/berkshelf-8.0.7/lib/berkshelf.rb:222:in `'
from /opt/cinc/embedded/lib/ruby/gems/3.1.0/gems/berkshelf-8.0.7/lib/berkshelf/cli.rb:1:in `require_relative'
from /opt/cinc/embedded/lib/ruby/gems/3.1.0/gems/berkshelf-8.0.7/lib/berkshelf/cli.rb:1:in `'
from :85:in `require'
from :85:in `require'
from /opt/cinc/embedded/lib/ruby/gems/3.1.0/gems/berkshelf-8.0.7/bin/berks:3:in `'
from /opt/cinc/embedded/bin/berks:25:in `load'
from /opt/cinc/embedded/bin/berks:25:in `'
```
## Affected versions
This issue impacts ParallelCluster versions from 3.8.0 to 3.9.3, across all OSes, schedulers.
## Mitigation
You can find a detailed explanation and the mitigation of the problem [in Wiki here](https://github.com/aws/aws-parallelcluster/wiki/(3.8.0-%E2%80%90-3.9.3)-ParallelCluster-Build-Image-Failing-during-Installation-of-Minitar-Ruby-Gem-Dependency)
Contributor guide
Research direction
Start by reproducing the failure with pcluster build-image and reviewing the linked mitigation wiki. Trace the LoadError from the Berkshelf packager.rb path and inspect the exported image logs. Done means affected ParallelCluster image builds complete without the archive/tar/minitar LoadError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, ruby
- Domain
- cloud, devops
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100