aws / aws/aws-parallelcluster

(3.8.0 - 3.9.3) ParallelCluster Build Image Failing during Installation of Minitar Ruby Gem Dependency

Open
#6,405 3 comments 2 reactions 0 assignees View on GitHub
3.x Known Issue Mitigated
Dominant language
Python
Stars
888
Forks
314
Avg merge
1d 10h
Merged PRs (30d)
43

Description

## The issue

During `pcluster build-image`, ParallelCluster installs Berkshelf gems for dependency management. `Minitar` is a dependency of Berkshelf. On Tuesday, August 6, Minitar released 2 different versions, v1.0.0 and last version of minitar v0.12 with the deprecated archive-tar-minitar v0.12.

The latest minitar v1.0.0 removed the `Archive::Tar` namespace and `archive/tar` path as they have been deprecated since 2017. And moved the executable in a separate `minitar-cli` package. These changes in `minitar` v 1.0.0+ causes failure in Berkshelf installation and hence the ParallelCluster Build Image fails.

Reference to Berkshelf Installation Failure https://github.com/chef/berkshelf/issues/26

## Error details

ParallelCluster fails to build images. The error surfaced by ParallelCluster Build Image is:

```
pcluster describe-image --image-id $IMAGE_ID | jq ".imagebuilderImageStatusReason"

Error occurred during operation 'Workflow Execution ID: 'wf-xxxxxx-xxxxxx-xxxxxx-xxxxxx-xxxxxx' failed with reason: Document arn::imagebuilder:::component/parallelclusterimage-*//1 failed!'
```

And, below is the snippet of the failure in ParallelCluster Build Image Logs which you can capture using [pcluster export-image-logs - AWS ParallelCluster](https://docs.aws.amazon.com/parallelcluster/latest/ug/pcluster.export-image-logs-v3.html)

```
LANG=en_US.UTF-8 sudo /opt/cinc/embedded/bin/berks vendor /etc/chef/cookbooks --delete || (echo 'Vendoring cookbook failed.' && exit 1)
done;
:85:in `require': cannot load such file -- archive/tar/minitar (LoadError)
from :85:in `require'
from /opt/cinc/embedded/lib/ruby/gems/3.1.0/gems/berkshelf-8.0.7/lib/berkshelf/packager.rb:1:in `'
from /opt/cinc/embedded/lib/ruby/gems/3.1.0/gems/berkshelf-8.0.7/lib/berkshelf/berksfile.rb:1:in `require_relative'
from /opt/cinc/embedded/lib/ruby/gems/3.1.0/gems/berkshelf-8.0.7/lib/berkshelf/berksfile.rb:1:in `'
from /opt/cinc/embedded/lib/ruby/gems/3.1.0/gems/berkshelf-8.0.7/lib/berkshelf.rb:222:in `require_relative'
from /opt/cinc/embedded/lib/ruby/gems/3.1.0/gems/berkshelf-8.0.7/lib/berkshelf.rb:222:in `'
from /opt/cinc/embedded/lib/ruby/gems/3.1.0/gems/berkshelf-8.0.7/lib/berkshelf/cli.rb:1:in `require_relative'
from /opt/cinc/embedded/lib/ruby/gems/3.1.0/gems/berkshelf-8.0.7/lib/berkshelf/cli.rb:1:in `'
from :85:in `require'
from :85:in `require'
from /opt/cinc/embedded/lib/ruby/gems/3.1.0/gems/berkshelf-8.0.7/bin/berks:3:in `'
from /opt/cinc/embedded/bin/berks:25:in `load'
from /opt/cinc/embedded/bin/berks:25:in `'
```

## Affected versions

This issue impacts ParallelCluster versions from 3.8.0 to 3.9.3, across all OSes, schedulers.

## Mitigation

You can find a detailed explanation and the mitigation of the problem [in Wiki here](https://github.com/aws/aws-parallelcluster/wiki/(3.8.0-%E2%80%90-3.9.3)-ParallelCluster-Build-Image-Failing-during-Installation-of-Minitar-Ruby-Gem-Dependency)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the failure with pcluster build-image and reviewing the linked mitigation wiki. Trace the LoadError from the Berkshelf packager.rb path and inspect the exported image logs. Done means affected ParallelCluster image builds complete without the archive/tar/minitar LoadError.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, ruby
Domain
cloud, devops
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.