bazelbuild / bazelbuild/rules_pkg

[TarFile]: normalize_path does not normalize when --directory is present

Open Beginner friendly
#1,072 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Starlark
Stars
253
Forks
221
Avg merge
10h 33m
Merged PRs (30d)
1

Description

### Problem description

When using `--directory` (`self.directory`) with `TarFile`, the following segment produces non-normalized path components:
```python
# pkg/private/tar/build_tar.py
# This prevents a potential problem for users with both a prefix_dir and
# symlinks that also repeat the prefix_dir. The old behavior was that we
# would get just the symlink path. Now we are prefixing with the prefix,
# so you get the file in the wrong place.
# We silently de-dup that. If people come up with a real use case for
# the /a/b/a/b/rest... output we can start an issue and come up with a
# solution at that time.
if self.directory and not dest.startswith(self.directory):
dest = self.directory + dest
```

We are using `--directory`, and so I was seeing paths like this:
```console
runtime/_main/../_main~ps~av_pip_attrs_3_10/pypi__attrs/attr/__init__.pyi
runtime/_main/../_main~ps~av_pip_attrs_3_10/pypi__attrs/attr/_cmp.pyi
runtime/_main/../_main~ps~av_pip_attrs_3_10/pypi__attrs/attr/_typing_compat.pyi
runtime/_main/../_main~ps~av_pip_attrs_3_10/pypi__attrs/attr/_version_info.pyi
runtime/_main/../_main~ps~av_pip_attrs_3_10/pypi__attrs/attr/converters.pyi
runtime/_main/../_main~ps~av_pip_attrs_3_10/pypi__attrs/attr/exceptions.pyi
runtime/_main/../_main~ps~av_pip_attrs_3_10/pypi__attrs/attr/filters.pyi
runtime/_main/../_main~ps~av_pip_attrs_3_10/pypi__attrs/attr/py.typed
runtime/_main/../_main~ps~av_pip_attrs_3_10/pypi__attrs/attr/setters.pyi
...
```
Note that these collapse into `runtime/_main~ps~av_pip_attrs_3_10/pypi__attrs/...`.

This may normally not be a problem, but becomes an issue when piping the `tar` file into `mksquashfs`:
```bash
Parallel mksquashfs: Using 32 processors
Creating 4.0 filesystem on bazel-out/k8-fastbuild/.squashfs, block size 131072.
[=======================| ] 1911/15662 12%
FATAL ERROR: Error: Tar pathname can't have '.' or '..' in it
```

### What to do

Call the existing `normpath()` on the concatenated path.

Contributor guide

Open the contributing guide

Research direction

Open pkg/private/tar/build_tar.py and inspect the --directory path handling around the existing normpath() call and concatenation. Reproduce the reported path shape with --directory, then apply the requested normalization to the concatenated path. Done means generated tar paths no longer contain '.' or '..' components and can be piped to mksquashfs without this pathname error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.