rsync breaks on excluded directories

Hi,

I understand that the gsutil utility cannot handle some file types and aborts. What I am trying to figure out is why this still occurs if I exclude the directory where those bad files exist. See below:

I set up two directories for this test: /tmp/test and /tmp/testdir. Both directories have a file called test.txt and a directory called skip:

ls -la /tmp/test
total 28
drwxr-xr-x. 3 btsherid lbg 34 Nov 6 09:06 .
drwxrwxrwt. 114 root root 12288 Nov 6 09:06 ..
drwxr-xr-x. 4 root root 4096 Nov 6 09:00 skip
-rw-r--r--. 1 root root 23 Nov 6 09:06 test.txt

ls -la /tmp/testdir
total 20
drwxr-xr-x. 3 root root 34 Nov 6 09:05 .
drwxrwxrwt. 114 root root 12288 Nov 6 09:06 ..
drwxr-xr-x. 2 root root 22 Nov 6 09:05 skip
-rw-r--r--. 1 root root 23 Nov 6 09:05 test.txt

The difference is that /tmp/test/skip contains files that I know will break gsutil, /tmp/testdir/skip just contains another text file. So I attempt to exclude the skip directory from copying but rsync still breaks for /tmp/test. I know that the exclude works correctly because /tmp/testdir does not copy the "skip" directory

gsutil -m rsync -r -e -d -C -x skip /tmp/testdir/ gs://lccc-gcp-archive/testdir/
Building synchronization state...
Starting synchronization...
Copying file:///tmp/testdir/test.txt [Content-Type=text/plain]...
/ [1/1 files][ 23.0 B/ 23.0 B] 100% Done
Operation completed over 1 objects/23.0 B.

gsutil -m rsync -r -e -d -C -x skip /tmp/test/ gs://lccc-gcp-archive/test/
Building synchronization state...
Caught non-retryable exception while listing file:///tmp/test/: 'ascii' codec can't decode byte 0xc4 in position 1: ordinal not in range(128)
CommandException: Caught non-retryable exception - aborting rsync

So if I am excluding the folder containing the invalid files, why does rsync still fail? I would think I should be able to exclude folders that I know will cause errors and then have rsync succeed on the rest of the directory.

Thanks,
Brendan

Start with the gsutil rsync command and reproduce the two examples using /tmp/testdir and /tmp/test, including the -x skip exclusion. Compare the directory-listing behavior and the ASCII decoding failure; done means excluded directories are not traversed or decoded and the remaining files synchronize successfully.

Written by the indexing model from the issue text.