git-ftp / git-ftp/git-ftp

git-ftp not working on commits that are deleting files (error message "fatal: Could not upload files., exiting...")

Open
#655 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Shell
Stars
5.6k
Forks
694
PR merge metrics
No merged PRs in 30d

Description

Bug / Enhancement / Question ?

Trying to run git ftp init (on a local Git repository where an old version of git ftp had already been used a couple of years ago, but this time uploading to a new sftp server - i.e. which doesn't have the .git-ftp.log file)

Immediately after displaying Uploading ..., it says : fatal: Could not upload files., exiting...

Trying to switch to another git branch, and reducing the number of files to a reasonable amount (200 files buffered for upload instead of 6672) didn't help to resolve the problem : still saying fatal: Could not upload files., exiting...

Running git ftp -vv init for verbose output, it says:

[…]
Jeu 20 avr 2023 13:27:02 CEST: Uploading ...
*   Trying 195.***.***.90:50027...
* Connected to 195.***.***.90 (195.***.***.90) port 50027 (#0)
* Found host 195.***.***.90 in /Users/pierre/.ssh/known_hosts
* Set "rsa-sha2-256,rsa-sha2-512,ssh-rsa" as SSH hostkey type
* SSH MD5 public key: NULL                                                                                                          
* SSH SHA256 public key: NULL
* SSH host check: 0, key: AAAAB3NzaC1y.....sKlkLyOKeK4lVHw==
* SSH authentication methods available: publickey,gssapi-keyex,gssapi-with-mic,password                                             
* Using SSH private key file ''
* SSH public key authentication failed: Unable to extract public key from private key file: Unable to open private key file
* Initialized password authentication
* Authentication complete
} [595 bytes data]                                                                                                                  
* We are completely uploaded and fine
############################################################################################################################## 100.0%* Connection #0 to host 195.***.***.90 left intact

* Found bundle for host: 0x7fee56422230 [serially]
* Re-using existing connection #0 with host 195.***.***.90
} [17497 bytes data]
* We are completely uploaded and fine
############################################################################################################################## 100.0%* Connection #0 to host 195.***.***.90 left intact

* Found bundle for host: 0x7fee56422230 [serially]
* Re-using existing connection #0 with host 195.***.***.90
} [30000 bytes data]
* We are completely uploaded and fine
############################################################################################################################## 100.0%* Connection #0 to host 195.***.***.90 left intact

* Found bundle for host: 0x7fee56422230 [serially]
* Re-using existing connection #0 with host 195.***.***.90
} [30000 bytes data]
* We are completely uploaded and fine
############################################################################################################################## 100.0%* Connection #0 to host 195.***.***.90 left intact

* Found bundle for host: 0x7fee56422230 [serially]
* Re-using existing connection #0 with host 195.***.***.90
} [4012 bytes data]
* We are completely uploaded and fine
############################################################################################################################## 100.0%* Connection #0 to host 195.***.***.90 left intact

* Found bundle for host: 0x7fee56422230 [serially]
* Re-using existing connection #0 with host 195.***.***.90
} [2326 bytes data]
* We are completely uploaded and fine
############################################################################################################################## 100.0%* Connection #0 to host 195.***.***.90 left intact

* Found bundle for host: 0x7fee56422230 [serially]
* Re-using existing connection #0 with host 195.***.***.90
} [2326 bytes data]
* We are completely uploaded and fine
############################################################################################################################## 100.0%* Connection #0 to host 195.***.***.90 left intact

curl: (3) URL using bad/illegal format or missing URL
Jeu 20 avr 2023 13:27:03 CEST: fatal: Could not upload files., exiting...

=> it would be nice to have more details about the bad/illegal format or missing URL (it's a pity that the git ftp -vv doesn't display the curl commands with their parameters)

Also tried with git ftp init --insecure but it's the same.

So I manually uploaded all the 200 files from this git branch, and then ran git ftp catchup, then switched back to my master git branch containing 6000+ files to be uploaded, and ran git ftp push: it displayed Uploading ... then it took several minutes to upload the files, and after a while, it stops unexpectedly, saying fatal: Could not upload files., exiting...

When running git ftp push -vv I get:

[…]
* Found bundle for host: 0x7f979899f880 [serially]
* Re-using existing connection #0 with host 195.***.***.90
} [598 bytes data]
* We are completely uploaded and fine
################################################################################################################################################################################################ 100.0%* Connection #0 to host 195.***.***.90 left intact

* Found bundle for host: 0x7f979899f880 [serially]
* Re-using existing connection #0 with host 195.***.***.90
} [1573 bytes data]
* We are completely uploaded and fine
################################################################################################################################################################################################ 100.0%* Connection #0 to host 195.***.***.90 left intact

* Found bundle for host: 0x7f979899f880 [serially]
* Re-using existing connection #0 with host 195.***.***.90
} [1096 bytes data]
* We are completely uploaded and fine
################################################################################################################################################################################################ 100.0%* Connection #0 to host 195.***.***.90 left intact

* Found bundle for host: 0x7f979899f880 [serially]
* Re-using existing connection #0 with host 195.***.***.90
} [937 bytes data]
* We are completely uploaded and fine
################################################################################################################################################################################################ 100.0%* Connection #0 to host 195.***.***.90 left intact

* Found bundle for host: 0x7f979899f880 [serially]
* Re-using existing connection #0 with host 195.***.***.90
} [415 bytes data]
* We are completely uploaded and fine
################################################################################################################################################################################################ 100.0%* Connection #0 to host 195.***.***.90 left intact

* Found bundle for host: 0x7f979899f880 [serially]
* Re-using existing connection #0 with host 195.***.***.90
} [5954 bytes data]
* We are completely uploaded and fine
################################################################################################################################################################################################ 100.0%* Connection #0 to host 195.***.***.90 left intact

curl: (3) URL using bad/illegal format or missing URL
Jeu 20 avr 2023 14:59:29 CEST: fatal: Could not upload files., exiting...

When connecting to the sftp server, I can see that it has successfully uploaded some of the files, but not all the files.

Environment
/bin/bash
Darwin BCL-MacBookPro-2.local 19.6.0 Darwin Kernel Version 19.6.0: Tue Jun 21 21:18:39 PDT 2022; root:xnu-6153.141.66~1/RELEASE_X86_64 x86_64
git-ftp version 1.6.0

Notes:

  • git-ftp has been installed through brew install git-ftp ; which in turns required dependency zstd ; which I had to install without support of LibLZMA because I'm running under macOS Catalina 10.15.7 (is this a problem ? Does git-ftp requires LibLZMA ??)
  • curl --version says :
Release-Date: 2023-03-20
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No source file or test is named. Start by reproducing the failure with git ftp init and git ftp push on commits containing deletions, using -vv to inspect the curl output. Done means the offending URL or deleted path is identified and the command reports a useful error while completing the valid uploads.

Written by the indexing model from the issue text.

Assessment

Tech stack
git, shell
Domain
cli, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.