CONP-PCNO / CONP-PCNO/git-annex-remote-globus
does nohow detect that downloaded content is actually an HTML requiring to authenticate
- Dominant language
- Python
- Stars
- 3
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
Files is there and I am even logged in into globus (unrelated to the remote mechanism):
```
$> globus ls 558c54fe-cc41-11e8-8c6a-0a1d4c5c824a:/SenzaiY/YMV01/YMV01_170818/ | grep ampli
amplitudes.npy
```
remote seems to happily download the file (although not sure what that last ExitFailure 1 about -- exit status seems clean):
```shell
$> git annex addurl --debug globus://558c54fe-cc41-11e8-8c6a-0a1d4c5c824a/SenzaiY/YMV01/YMV01_170818/amplitudes.npy
[2020-06-29 22:43:43.768509247] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","git-annex"]
[2020-06-29 22:43:43.773170539] process done ExitSuccess
[2020-06-29 22:43:43.773271196] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","show-ref","--hash","refs/heads/git-annex"]
[2020-06-29 22:43:43.777702694] process done ExitSuccess
[2020-06-29 22:43:43.777907788] read: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","log","refs/heads/git-annex..3458fdc310d2deaeaa80c8dfaefe1698a9682e1c","--pretty=%H","-n1"]
[2020-06-29 22:43:43.790929834] process done ExitSuccess
[2020-06-29 22:43:43.791368807] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch"]
[2020-06-29 22:43:43.791908396] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","cat-file","--batch-check=%(objectname) %(objecttype) %(objectsize)"]
[2020-06-29 22:43:43.796621346] chat: /home/yoh/proj/datalad/git-annex-remote-globus/venvs/dev3/bin/git-annex-remote-globus []
[2020-06-29 22:43:44.238477469] git-annex-remote-globus[1] --> VERSION 1
[2020-06-29 22:43:44.2387346] git-annex-remote-globus[1] <-- EXTENSIONS INFO
[2020-06-29 22:43:44.239000521] git-annex-remote-globus[1] --> EXTENSIONS
[2020-06-29 22:43:44.239135367] git-annex-remote-globus[1] <-- PREPARE
[2020-06-29 22:43:44.2393114] git-annex-remote-globus[1] --> GETCONFIG uuid
[2020-06-29 22:43:44.239419178] git-annex-remote-globus[1] <-- VALUE 558c54fe-cc41-11e8-8c6a-0a1d4c5c824a
[2020-06-29 22:43:44.239597611] git-annex-remote-globus[1] --> GETCONFIG fileprefix
[2020-06-29 22:43:44.239709186] git-annex-remote-globus[1] <-- VALUE
[2020-06-29 22:43:44.239849382] git-annex-remote-globus[1] --> GETCONFIG endpoint
[2020-06-29 22:43:44.239962534] git-annex-remote-globus[1] <-- VALUE 558c54fe-cc41-11e8-8c6a-0a1d4c5c824a
[2020-06-29 22:43:44.610854782] git-annex-remote-globus[1] --> PREPARE-SUCCESS
[2020-06-29 22:43:44.611090293] git-annex-remote-globus[1] <-- CLAIMURL globus://558c54fe-cc41-11e8-8c6a-0a1d4c5c824a/SenzaiY/YMV01/YMV01_170818/amplitudes.npy
[2020-06-29 22:43:44.611351451] git-annex-remote-globus[1] --> CLAIMURL-SUCCESS
[2020-06-29 22:43:44.611508727] git-annex-remote-globus[1] <-- CHECKURL globus://558c54fe-cc41-11e8-8c6a-0a1d4c5c824a/SenzaiY/YMV01/YMV01_170818/amplitudes.npy
[2020-06-29 22:43:44.875955308] git-annex-remote-globus[1] --> CHECKURL-CONTENTS 13950872
addurl globus://558c54fe-cc41-11e8-8c6a-0a1d4c5c824a/SenzaiY/YMV01/YMV01_170818/amplitudes.npy (from globus) (to 558c54fe_cc41_11e8_8c6a_0a1d4c5c824a_SenzaiY_YMV01_YMV01_170818_amplitudes.npy) [2020-06-29 22:43:44.876574066] read: git ["--version"]
[2020-06-29 22:43:44.880568781] process done ExitSuccess
[2020-06-29 22:43:44.880762238] chat: git ["--git-dir=.git","--work-tree=.","check-ignore","-z","--stdin","--verbose","--non-matching"]
[2020-06-29 22:43:44.909946917] git-annex-remote-globus[1] <-- TRANSFER RETRIEVE URL-s13950872--globus://558c54fe-cc41-11e8-8c6-1d0e3f16a2fbf978cc33ec614c02e79d .git/annex/tmp/URL-s13950872--globus&c%%558c54fe-cc41-11e8-8c6-1d0e3f16a2fbf978cc33ec614c02e79d
[2020-06-29 22:43:44.910314097] git-annex-remote-globus[1] --> GETURLS URL-s13950872--globus://558c54fe-cc41-11e8-8c6-1d0e3f16a2fbf978cc33ec614c02e79d globus://
[2020-06-29 22:43:44.912313981] git-annex-remote-globus[1] <-- VALUE globus://558c54fe-cc41-11e8-8c6a-0a1d4c5c824a/SenzaiY/YMV01/YMV01_170818/amplitudes.npy
[2020-06-29 22:43:44.912479181] git-annex-remote-globus[1] <-- VALUE
[2020-06-29 22:43:46.3695805] git-annex-remote-globus[1] --> TRANSFER-SUCCESS RETRIEVE URL-s13950872--globus://558c54fe-cc41-11e8-8c6-1d0e3f16a2fbf978cc33ec614c02e79d
[2020-06-29 22:43:46.370252941] chat: git ["--git-dir=.git","--work-tree=.","--literal-pathspecs","check-attr","-z","--stdin","annex.backend","annex.numcopies","annex.largefiles","--"]
[2020-06-29 22:43:46.37075713] read: git ["--version"]
[2020-06-29 22:43:46.374468113] process done ExitSuccess
(non-large file; adding content to git repository) ok
(recording state in git...)
[2020-06-29 22:43:46.393727743] feed: xargs ["-0","git","--git-dir=.git","--work-tree=.","--literal-pathspecs","add","--"]
[2020-06-29 22:43:46.443980111] process done ExitSuccess
[2020-06-29 22:43:46.512114758] process done ExitSuccess
[2020-06-29 22:43:46.512947576] process done ExitSuccess
[2020-06-29 22:43:46.51342478] process done ExitSuccess
[2020-06-29 22:43:46.513917556] process done ExitSuccess
[2020-06-29 22:43:46.514354977] process done ExitFailure 1
(dev3) 2 19497 [1].....................................:Mon 29 Jun 2020 10:43:47 PM EDT:.
(git)smaug:/tmp/testdsg[master]git
$> echo $?
0
```
but that file is actually an HTML demanding the login -- may be the credentials record didn't workout or smth like that -- dont' know:
```
$> file -L 558c54fe_cc41_11e8_8c6a_0a1d4c5c824a_SenzaiY_YMV01_YMV01_170818_amplitudes.npy
558c54fe_cc41_11e8_8c6a_0a1d4c5c824a_SenzaiY_YMV01_YMV01_170818_amplitudes.npy: HTML document, UTF-8 Unicode text, with very long lines
$> grep 'login' 558c54fe_cc41_11e8_8c6a_0a1d4c5c824a_SenzaiY_YMV01_YMV01_170818_amplitudes.npy
Additional logins needed to use 2a9f4.8443.dn.glob.us
Use your existing organizational login
Globus ID is an identity provider operated by Globus. If you don't want to use your existing organizational login (e.g. university, national lab, facility, project, Google) with Globus, you can create an account on Glob
us ID and use that to log into Globus.
Continue
var continue_btn = $('#login-btn');
/* Load / save the selected IdP in localStorage for simple login convenience */
```
I must confess that the remote configuration is also quite ad-hoc since I see no point in all of those endpoint specific settings to be able to download urls (which already carry that information) but they are needed for current code to function:
```
$> git show git-annex:remote.log
558c54fe-cc41-11e8-8c6a-0a1d4c5c824a encryption=none endpoint=558c54fe-cc41-11e8-8c6a-0a1d4c5c824a externaltype=globus name=globus type=external uuid=558c54fe-cc41-11e8-8c6a-0a1d4c5c824a timestamp=1593482271.014958164s
```
but the point is that the code should detect that I am not properly authenticated
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the git-annex-remote-globus entry point and reproduce the globus:// transfer with the shown git annex addurl command. Inspect how the downloaded response is validated after authentication fails; done means an HTML login response is rejected and reported as an authentication error rather than recorded as the requested file.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git, python
- Domain
- cli, tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100