galaxyproject / galaxyproject/planemo

planemo shed_update --check_diff --> gzip error from Tool Shed error

Open
#624 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
110
Forks
102
Avg merge
4d 21h
Merged PRs (30d)
13

Description

```
$ planemo --version
planemo, version 0.38.0.dev0
```

```
$ planemo shed_update --shed_target testtoolshed packages/package_mira_4_0_2/
cd '/mnt/shared/users/pc40583/repositories/pico_galaxy/packages/package_mira_4_0_2' && git rev-parse HEAD
cd '/mnt/shared/users/pc40583/repositories/pico_galaxy/packages/package_mira_4_0_2' && git diff --quiet
Could not update package_mira_4_0_2
Unexpected HTTP status code: 400: {"content_alert": "", "err_msg": "No changes to repository."}
Repository metadata updated successfully for repository package_mira_4_0_2.
Failed to update a repository.
```

Now using ``--check_diff``,

```
$ planemo shed_update --check_diff --shed_target testtoolshed packages/package_mira_4_0_2/
cd '/mnt/shared/users/pc40583/repositories/pico_galaxy/packages/package_mira_4_0_2' && git rev-parse HEAD
cd '/mnt/shared/users/pc40583/repositories/pico_galaxy/packages/package_mira_4_0_2' && git diff --quiet
Diffing repository [package_mira_4_0_2]
wget -q -O - 'https://testtoolshed.g2.bx.psu.edu/repository/download?repository_id=6777f9675c9051d3&changeset_revision=default&file_type=gz' | tar -xzf - -C /tmp/tool_shed_diff_YeKVAJ/_testtoolshed_ --strip-components 1

gzip: stdin: invalid compressed data--format violated
tar: Child returned status 1
tar: Error is not recoverable: exiting now
mkdir "/tmp/tool_shed_diff_YeKVAJ/_workingdir_"; tar -xzf "/tmp/tmpfuXDVs" -C "/tmp/tool_shed_diff_YeKVAJ/_workingdir_"; rm -rf /tmp/tmpfuXDVs
cd "/tmp/tool_shed_diff_YeKVAJ"; diff -r _workingdir_ _testtoolshed_
Repository [package_mira_4_0_2] not different, skipping upload.
Repository metadata updated successfully for repository package_mira_4_0_2.
```

What is this message and is it hiding a problem? It turns out to be something on the (Test) Tool Shed side:

```
$ wget -q -O - 'https://testtoolshed.g2.bx.psu.edu/repository/download?repository_id=6777f9675c9051d3&changeset_revision=default&file_type=gz' | head
UX?




body { color: #303030; background: #dfe5f9; font-family:"Lucida Grande",verdana,arial,helvetica,sans-serif; font-size:12px; line-height:16px; }
.content { max-width: 720px; margin: auto; margin-top: 50px; }

Internal Server Error
```

Verbose mode (e.g. with curl) shows the initial URL returns a 302 redirection, which ultimately gives an error message but not an error status:

```
$ curl -v 'https://testtoolshed.g2.bx.psu.edu/repos/peterjc/package_mira_4_0_2/archive/default.tar.gz'
* About to connect() to testtoolshed.g2.bx.psu.edu port 443 (#0)
* Trying 128.118.250.5... connected
* Connected to testtoolshed.g2.bx.psu.edu (128.118.250.5) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* SSL connection using TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
* Server certificate:
* subject: CN=radegast.galaxyproject.org,OU=Center for Comparative Genomics and Bioinformatics,O=The Pennsylvania State University,L=University Park,ST=PA,postalCode=16802,C=US
* start date: Nov 11 00:00:00 2016 GMT
* expire date: Nov 11 23:59:59 2017 GMT
* common name: radegast.galaxyproject.org
* issuer: CN=InCommon RSA Server CA,OU=InCommon,O=Internet2,L=Ann Arbor,ST=MI,C=US
> GET /repos/peterjc/package_mira_4_0_2/archive/default.tar.gz HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/3.21 Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: testtoolshed.g2.bx.psu.edu
> Accept: */*
>
< HTTP/1.1 200 Script output follows
< Server: nginx/1.6.2
< Date: Wed, 01 Feb 2017 17:49:32 GMT
< Content-Type: application/x-gzip
< Transfer-Encoding: chunked
< Connection: keep-alive
< ETag: 1485971064
< Content-Disposition: attachment; filename=package_mira_4_0_2-default.tar.gz
<
UX?




body { color: #303030; background: #dfe5f9; font-family:"Lucida Grande",verdana,arial,helvetica,sans-serif; font-size:12px; line-height:16px; }
.content { max-width: 720px; margin: auto; margin-top: 50px; }

Internal Server Error




Internal Server Error

Galaxy was unable to successfully complete your request


An error occurred.

This may be an intermittent problem due to load or other unpredictable factors, reloading the page may address the problem.

The error has been logged to our team.




* Connection #0 to host testtoolshed.g2.bx.psu.edu left intact
* Closing connection #0
```

Contributor guide

Open the contributing guide

Research direction

Reproduce the report with `planemo shed_update --check_diff --shed_target testtoolshed packages/package_mira_4_0_2/` and inspect the `shed_update --check_diff` entry point that downloads and extracts the Tool Shed archive. Determine how an HTTP error page returned with a success status is handled; done means the command reports the remote failure clearly without misleading gzip or diff output.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.