common-workflow-language / common-workflow-language/cwltool

Capture last modification date of remote file

Open
#1,672 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
376
Forks
255
Avg merge
2d 7h
Merged PRs (30d)
12

Description

Hello,

Would it be possible to implement automatically retaining the last modification date of a remote file in cwltool, instead of setting the download timestamp as last modification date?

When I download a file in a CWL workflow run using an explicit download step (using `wget` or `curl --remote-time`) the modification timestamp of the remote file is carried over to the downloaded file. However, this is not the case when I skip the download step and instead specify the input for the next step as the remote file itself: here the download date is set as last modification date (see https://github.com/RenskeW/cwlprov-provenance/tree/83e230bff19501fbc461c3d93fac9d9c1432cc53/modification_date for a demonstration).

Considering that last modification date may be of interest from a provenance perspective, it could be really helpful if cwltool also captured the remote timestamp by default (possibly in addition to the download date).

## Expected Behavior
```
file_to_download:
class: File
location: https:// # has last modification date
```
should result in:
A downloaded file with the same modification timestamp as on the remote location, instead of the download date

## Actual Behavior
See https://github.com/RenskeW/cwlprov-provenance/tree/83e230bff19501fbc461c3d93fac9d9c1432cc53/modification_date

## Your Environment
* cwltool version: 3.1.20220502060230

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.