nf-core / nf-core/website

Pipeline conflicts with user AWS configuration

Open
#355 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

documentation infrastructure
Dominant language
MDX
Stars
98
Forks
298
Avg merge
14h 38m
Merged PRs (30d)
38

Description

I’ve got a default AWS configuration at $HOME/.aws/config which includes the following:

[default]
region = us-east-1

This causes Sarek to fail when it’s trying to download the iGenomes data. The error message is:

The authorization header is malformed; the region 'us-east-1' is wrong; expecting 'eu-west-1' (Service: Amazon S3; Status Code: 400; Error Code: AuthorizationHeaderMalformed; Request ID: EFD0047C3BC01410; S3 Extended Request ID: MMiKH3WnN3XQ8ZdtQN48D6lriG36JRv7cbMm5yuNv8cNqHpwhfB5XFUJvZt4pY6qX1ZsOQoPGMg=)

-- Check script '/home/konrad/.nextflow/assets/nf-core/sarek/main.nf' at line: 268 or see '.nextflow.log' file for more details

Removing the default AWS profile fixes the issue, but should not be necessary: Sarek should explicitly set its own AWS profile (including a region) to avoid conflicts with system profiles.

(This is using Sarek at rev b952fe2b3fcd3541237c9f4a9f27e1852f537967 with no user configuration, launched as nextflow run nf-core/sarek --input samples.)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with nextflow run nf-core/sarek --input samples while $HOME/.aws/config sets region = us-east-1, then inspect main.nf at line 268 and .nextflow.log. Trace how the iGenomes download selects its AWS region or profile. Done means the pipeline downloads iGenomes without requiring removal of the user's default AWS profile.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws
Domain
cloud, devops
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.