zstd -z f1 -1 --patch-from f2` is 20% slower than concatenating both files and compressing

**Describe the bug**
I created a random 32KB file, and another one which is 50% identical and 50% random (in random offsets).
Then, when trying to compress the second one as a patch from the first one, I received a 20% larger latency when compared to simply concatenating both and compressing together.

I don't understand why this would happen.

**To Reproduce**
Steps to reproduce the behavior:
1. Create 32KB random file: `reference_chunk`

2. Create 32KB random file `using_reference_chunk`, and then for each 1KB block in the file, copy the same data (from the
offsets) from `reference_chunk` with 50% probability.

3. Run the following script to compress in both methods:
```
cat reference_chunk using_reference_chunk > both_chunks
for i in `seq 10000`; do \
rm *.zst; \
perf stat zstd -z -1 both_chunks 2> >(grep seconds | cut -d" " -f8); \
rm *.zst; \
perf stat zstd -z -1 using_reference_chunk --patch-from reference_chunk 2> >(grep seconds | cut -d" " -f8); \
done > sum_both
```

4. Analyze results with python3 script:
```
f = open('sum_both' ,'r')
lines = f.readlines()
lines = [float(x.strip()) for x in lines]
patch_mode = lines[1::2]
no_patch_mode = lines[::2]
print(f'no_patch: {sum(no_patch_mode)}')
print(f'patch: {sum(patch_mode)} (+{(100*sum(patch_mode)/sum(no_patch_mode) - 100):.05}%)')
print(f'tot: {sum(lines)}')
f.close()
```
5. Results:
```
$ python3 analysis_script.py
no_patch: 21.50548151100002
patch: 26.27372080199989 (+22.172%)
tot: 47.77920231299978
```

**Expected behavior**
I would expect both methods to be roughly identical.
When comparing the 2nd part of `both_chunks.zst` to `using_reference_chunk.zst` they are nearly identical (save a few different bytes around the header and footer).
(note that since the first 32KB are random then they weren't compressed, as expected).

**Desktop (please complete the following information):**
- OS: tested on MacOS and Redhat
- Version: Reproduced on 1.4.9 and 1.4.10
- Compiler: Reproduced on clang and gcc
- Flags [e.g O2]: Both with and without `-g`, built using makefile.

Attached are `reference_chunk, using_reference_chunk, both_chunks`.
[Archive.zip](https://github.com/facebook/zstd/files/6350600/Archive.zip)