astral-sh / astral-sh/tar-codec
Expose byte-preserving member paths and link targets when decoding
- Dominant language
- Rust
- Stars
- 9
- Forks
- 3
- Avg merge
- 9m
- Merged PRs (30d)
- 3
Description
## Summary
The high-level decoder currently converts effective member paths and link targets with `std::str::from_utf8` and returns `DecodeError::InvalidUtf8` when they are not UTF-8. This prevents consumers from decoding otherwise valid archives whose names or symlink targets are arbitrary byte sequences.
Unix filesystem names are byte strings (excluding NUL and `/`), and USTAR/GNU header fields can contain non-UTF-8 bytes. Some archive consumers need to validate and preserve those bytes without interpreting them as text.
## Suggested API direction
Please consider exposing effective paths and link targets in a byte-preserving form, either:
- by changing or extending `MemberMetadata` and link targets to use a byte-string type; or
- by providing a lower-level public member API that retains the effective raw bytes while keeping the existing UTF-8 format-neutral API.
The framing layer already computes effective paths and link targets as byte slices, so ideally the high-level projection would not make UTF-8 validation unavoidable. Extraction policy could continue to impose its own platform-appropriate name restrictions.
## Impact
Without such an API, applications that otherwise use byte-oriented filesystem models must reject non-UTF-8 archives or bypass `tar-codec`’s high-level decoder.
Contributor guide
Research direction
Start by tracing the high-level decoder's conversion of effective member paths and link targets through MemberMetadata and DecodeError::InvalidUtf8, then review how the framing layer exposes the underlying byte slices. The design is complete when consumers can preserve arbitrary valid bytes through a public API without making the existing UTF-8 format-neutral API or extraction policy unsafe.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100