Pulp export chunks are seen by suspicious by virus scanners because tar output is piped into split
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 598
- Forks
- 168
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 86
Description
Version
All
Describe the bug
See attached BZ for full details.
The gzipped tarball is piped into the split command to create the chunks. Every chunk ( or split ) after the first one does not have file header data to indicate the type of file, so the virus scanner sees it as random data.
When creating a chunked tar archive on my local machine and piping it through "split", only the first chunk is recognised with the file command:
woody@woody-arch-thinkpad ~/Documents $ tar -cvzf - file1.mp4 file2.mp4 | split -b 2MB - test.tar.gz.
woody@woody-arch-thinkpad ~/Documents $ file test.tar.gz.aa
test.tar.gz.aa: gzip compressed data, from Unix, original size modulo 2^32 842674519
woody@woody-arch-thinkpad ~/Documents $ file test.tar.gz.ab
test.tar.gz.ab: data
If I use the "--tape-length" option, I can use the "file" command to identify each chunk as a tarball. I assume the virus scanning tool would be using "file" or similar methods.
woody@woody-arch-thinkpad ~/Documents $ tar -cv --tape-length=2000 --file=test-archive-{0..100}.tar file1.mp4 file2.mp4
woody@woody-arch-thinkpad ~/Documents $ file test-archive-0.tar
test-archive-0.tar: POSIX tar archive (GNU)
woody@woody-arch-thinkpad ~/Documents $ file test-archive-1.tar
test-archive-1.tar: tar archive
A drawback is that those "--tape-length" archives can't be gzipped from that single tar command. It gives an error
"tar: Cannot use multi-volume compressed archives"
Can we request that the exports are chunked with "--tape-length" instead of piping into the "split" command?
Additional context
https://bugzilla.redhat.com/show_bug.cgi?id=2109742
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No source file or test is named in the issue. Start by locating the export implementation that pipes tar output through split, then reproduce the chunking behavior described with file and a virus scanner; done means exported chunks are individually recognized without breaking archive reconstruction.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100